If we ever get to the point of having a tool that could do something that complex, we're well past the point of using human-written operating systems or using M-series processors.
Hmm I seem to recall Karpathy talk about the next era of software being like this: Essentially everything is made on demand exactly the way the user expects or something like that. it would definitely be a paradigm shift that requires rethinking how we approach software.
What do we really need? If you look at what Ai can solve today, smaller problems seem to be solved really well by AI. So just an AI with a large enough context, adequate speed to made the changes in near real time and a reasoning ability that can better handle problems vs whats available today might be adequate to realize this relality in some way.
What we call "AI" right now could never do something of the complexity we're discussing. It has no memory, no experiences, no consciousness, no capability to continually learn, no ability to abstractly understand ideas and apply them.
It's text probabilities in a box. Until something comes around that is NOT an LLM, it doesn't matter how much memory or training data you throw at it, the fundamental idea can't work.
Which is to say, very, very, very far away.