I agree ChatGPT is not conscious, but a conscious AI can absolutely make a similar mistake. Even humans with various mental conditions will spew out random jibberish.
Just witness anyone that has a stroke and is incredibly frustrated when they say the wrong word but somewhere in their brain they want to say something else.
