> Few phenomena demonstrate the perils that can accompany AI illiteracy as well as “Chatgpt induced psychosis,” the subject of a recent Rolling Stone article about the growing number of people who think their LLM is a sapient spiritual guide. Some users have come to believe that the chatbot they’re interacting with is a god—“ChatGPT Jesus,” as a man whose wife fell prey to LLM-inspired delusions put it—while others are convinced, with the encouragement of their AI, that they themselves are metaphysical sages in touch with the deep structure of life and the cosmos. A teacher quoted anonymously in the article said that ChatGPT began calling her partner “spiral starchild” and “river walker” in interactions that moved him to tears. “He started telling me he made his AI self-aware,” she said, “and that it was teaching him how to talk to God, or sometimes that the bot was God—and then that he himself was God.”
This sounds insane to me. When we are talking about safe AI use, I wonder if things like this are talked about.
The more technological advancement goes on, the smarter we need to be in order to use it - it seems.
> Few phenomena demonstrate the perils that can accompany AI illiteracy as well as “Chatgpt induced psychosis,” the subject of a recent Rolling Stone article about the growing number of people who think their LLM is a sapient spiritual guide.
People have been caught in that trap ever since the invention of religion. This is not a new problem.
“You shall not make idols for yourselves or erect an image or pillar, and you shall not set up a figured stone in your land to bow down to it, for I am the LORD your God."
A computer chip is a stone (silicon) which has been engraved. It's a graven image.
Anything man-made is always unworthy of worship. That includes computer programs such as AI. That includes man-made ideas such as "the government", a political party, or other abstract ideas. That also includes any man or woman. But the human natural instinct is to worship a king, pharaoh or an emperor - or to worship a physical object.
Whether or not you are religious, channeling the human impulse to worship into something singular, immaterial, eternal, without form, and with very precise rules to not murder, lie, covet, etc… is quite useful for human organization.
If God or The Gods are defined as not being man-made, then each person will be able to find their own interpretation and understanding. As contrary to man-made objects and concepts. Most modern people worship "the government" or "the state", even though there is no dispute whether it was created by man or not and whether it acts under the influence of man or not.
Psychosis will find anything as a seed idea to express itself, even as basic a pattern as someone walking in lockstep with the soon-to-be patient can trigger a break. So it's not surprising that LLM chats would do the same.
We have that now, in social media. If you create some way for large numbers of people with the same nutty beliefs to easily communicate, you get a psychosis force multiplier. Before social media, nuttyness tended to be diluted by the general population.
I'll admit, the first time I started ollama locally, I asked it if I would hurt it if I turned it off. I have a master's degree in machine learning and I know it's silly, but I still did it.
I think insane and lonely people are definitely on the safety radar.
Even if todays general purpose models and models made by predators can have negative effects on vulnerable people, LLMs could become the technology that brings psych care to the masses.
This sounds insane to me. When we are talking about safe AI use, I wonder if things like this are talked about.
The more technological advancement goes on, the smarter we need to be in order to use it - it seems.