> If you ask ChatGPT how to use it, it will roleplay a character called “Assistant” from a counterfactual world where “how do I use Assistant?” has a single, well-defined answer. Because it is role-playing – improvising – it will not always give you the same answer. And none of the answers are true, about the real world. They’re about the fantasy world, where the fantasy app called “Assistant” really exists.

"Role playing" is a great analogy for what these models are doing. If an AI is asked a reasonable question it doesn't know the answer to, it won't go "I'm sorry, I'm not sure, let me go check," it performs improv by generating something that kinda feels like what the right answer might be

I'm not sure why it can't go check. I've tried to play with prompts like "if you're not sure, respond with a Google search query instead, and I'll paste the top result."

it feels like these language models could be a lot smaller if they knew how to Google like us.

I don't know much about AI, but does it know that it doesn't know something? Like would it be able to tell that it's unsure or even wrong?

Knowing is a human construct for a set of interactions in our brain, so I am not sure it is meaningful to ask if a LLM "knows" something.

Do the atoms that comprise our brain "know" or do they "interact"?

Because it wouldn't know to check.

An AI being right is just when the output of one of its roleplaying sessions is a true statement. AIs can have a version of "confidence", but it's not how true it thinks its statements are, it's how well it thinks it's role-playing.

It helps a bit, but isn’t perfect by any means. What is most useful is it keeps it on topic when it has a document in mind.

Relevant: I took Solr, Selenium and GPT3 and mushed ‘em together.

What if you don't use Google...

IANAAIA: why can't they do Morse code?