This is really cool, I was wondering how memory had been implemented in ChatGPT. Very interesting to see the completely different approaches. It seems to me like Claude's is better suited for solving technical tasks while ChatGPT's is more suited to improving casual conversation (and, as pointed out, future ads integration).
I think it probably won't be too long before these language-based memories look antiquated. Someone is going to figure out how to store and retrieve memories in an encoded form that skips the language representation. It may actually be the final breakthrough we need for AGI.
> It may actually be the final breakthrough we need for AGI.
I disagree. As I understand them, LLMs right now don’t understand concepts. They actually don’t understand, period. They’re basically Markov chains on steroids. There is no intelligence in this, and in my opinion actual intelligence is a prerequisite for AGI.
I don’t understand the argument “AI is just XYZ mechanism, therefore it cannot be intelligent”.
Does the mechanism really disqualify it from intelligence if behaviorally, you cannot distinguish it from “real” intelligence?
I’m not saying that LLMs have certainly surpassed the “cannot distinguish from real intelligence” threshold, but saying there’s not even a little bit of intelligence in a system that can solve more complex math problems than I can seems like a stretch.
> if behaviorally, you cannot distinguish it from “real” intelligence?
Current LLMs are a long way from there.
You may think "sure seems like it passes the Turing test to me!" but they all fail if you carry on a conversation long enough. AIs need some equivalent of neuroplasticity and as of yet they do not have it.
This is what I think is the next evolution of these models. Our brains are made up of many different types of neurones all interspersed with local regions made up of specific types. From my understanding most approaches to tensors don't integrate these different neuronal models at the node level; it's usually by feeding several disparate models data and combining an end result. Being able to reshape the underlying model and have varying tensor types that can migrate or have a lifetime seems exciting to me.
Strongly agree with this. When we were further from AGI, many people imagined that there is a single concept of AGI that would be obvious when we reached it. But now, we're close enough to AGI for most people to realize that we don't know where it is. Most people agree we're at least moving more towards it than away form it, but nobody knows where it is, and we're still too focused on finding it than making useful things.
Incorrect. Vertebrate animal brains update their neural connections when interacting with the environment. LLMs don't do that. Their model weights are frozen for every release.
But why can’t I then just say, actually, you need to relocate the analogy components; activations are their neural connections, the text is their environment, the weights are fixed just like our DNA is, etc.
As I understand it, octopuses have their reasoning and intelligence essentially baked into them at birth, shaped by evolution, and do relatively little learning during life because their lives are so short. Very intelligent, obviously, but very unlike people.
Maybe panpsychism is true and the machine actually does have a soul, because all machines have souls, even your lawnmower. But possibly the soul of a machine running a frontier AI is a bit closer to a human soul than your lawnmower’s soul is.
Scientifically, intelligence requires organizational complexity. And has for about a hundred years.
That does actually disqualify some mechanisms from counting as intelligent, as the behaviour cannot reach that threshold.
We might change the definition - science adapts to the evidence, but right now there are major hurdles to overcome before such mechanisms can be considered intelligent.
What is the scientific definition of intelligence? I assume that is it is comprehensive, internally consistent, and that it fits all of the things that are obviously intelligent and excludes the things which are obviously not intelligent. Of course being scientific I assume it is also falsifiable.
It can’t learn or think unless prompted, then it is given a very small slice of time to respond and then it stops. Forever. Any past conversations are never “thought” of again.
It has no intelligence. Intelligence implies thinking and it isn’t doing that. It’s not notifying you at 3am to say “oh hey, remember that thing we were talking about. I think I have a better solution!”
Just because it's not independent and autonomous does not mean it could not be intelligent.
If existing humans minds could be stopped/started without damage, copied perfectly, and had their memory state modified at-will would that make us not intelligent?
> Just because it's not independent and autonomous does not mean it could not be intelligent.
So to rephrase: it’s not independent or autonomous. But it can still be intelligent. This is probably a good time to point out that trees are independent and autonomous. So we can conclude that LLMs are possibly as intelligent as trees. Super duper.
> If existing humans minds could be stopped/started without damage, copied perfectly, and had their memory state modified at-will would that make us not intelligent?
To rephrase: if you take something already agreed to as intelligent, and changed it, is it still intelligent? The answer is, no damn clue.
These are worse than weak arguments, there is no thesis.
The thesis is that "intelligence" and "independence/autonomy" are independent concepts. Deciding whether LLMs have independence/autonomy does not help us decide if they are intelligent.
I think that’s a valid assessment of my argument, but it goes further than just “always on”. There’s an old book called On Intelligence that asked these kinds of questions 20+ years ago (of AI), I don’t remember the details, but a large part of what makes something intelligent doesn’t just boil down to what you know and how well you can articulate it.
For example, we as humans aren’t even present in the moment — different stimuli take different lengths of time to reach our brain, so our brain creates a synthesis of “now” that isn’t even real. You can’t even play Table Tennis unless you can predict up to one second in the future with enough details to be in the right place to hit the ball the ball before you hit the ball to your opponent.
Meanwhile, an AI will go off-script during code changes, without running it by the human. It should be able to easily predict the human is going to say “wtaf” when it doesn’t do what is asked, and handle that potential case BEFORE it’s an issue. That’s ultimately what makes something intelligent: the ability to predict the future, anticipate issues, and handle them.
>They’re basically Markov chains on steroids. There is no intelligence in this, and in my opinion actual intelligence is a prerequisite for AGI.
This argument is circular.
A better argument should address (given the LLM successes in many types of reasoning, passing the turing test, and thus at producing results that previously required intelligence) why human intelligence might not also just be "Markov chains on even better steroids".
Humans think even when not being prompted by other humans, and in some cases can learn new things by having intuition make a concept clear or by performing thought experiments or by combining memories of old facts and new facts across disciplines. Humans also have various kinds of reasoning (deductive, inductive, etc.). Humans also can have motivations.
I don’t know if AGI needs to have all human traits but I think a Markov chain that sits dormant and does not possess curiosity about itself and the world around itself does not seem like AGI.
>Humans think even when not being prompted by other humans
That's more of an implementation detail. Humans take constant sensory input and have some sort of way to re-introduce input later (e.g. remember something).
Both could be added (even trivially) to LLMs.
And it's not at all clear human thought is contant. It just appears so in our naive intuition (same how we see a movie as moving, not as 24 static frames per second). It's a discontinuous mechanism though (propagation time, etc), and this has been shown (e.g. EEG/MEG show the brain sample sensory input in a periodic pattern, stimuly with small time difference are lost - as if there is a blind-window regarding perception, etc).
>and in some cases can learn new things by having intuition make a concept clear or by performing thought experiments or by combining memories of old facts and new facts across disciplines
Unless we define intuition in a way that excludes LLM style mechanisms a priori, whose to say LLMs don't do all those things as well, even if in a simpler way?
They've been shown to combine stuff across disciplines, and also to develop concepts not directly on their training set.
And "performing thought experiments" is not that different than the reasoning steps and backtracking LLMs also already do.
Not saying LLMs are on parity with human thinking/consciousness. Just that it's not clear that they're doing more or less the same even at reduced capacity and with a different architecture and runtime setup.
The environment is constantly prompting you. That ad you see of Coca Cola is prompting you to do something. That hunger feeling is prompting “you” to find food. That memory that makes you miss someone is another prompt to find that someone - or to avoid.
Sometimes the prompt is outside your body other times is inside.
Roughly, actual intelligence needs to maintain a world model in its internal representation, not merely an embedding of language, which is a very different data structure and probably will be learned in a very different way. This includes things like:
- a map of the world, or concept space, or a codebase, etc
- causality
- "factoring" which breaks down systems or interactions into predictable parts
Language alone is too blurry to do any of these precisely.
It probably is a lot like that! I imagine it's a matter of specializing the networks and learning algorithms to converge to world-model-like-structures rather than language-like-ones. All these models do is approximate the underlying manifold structure, just, the manifold structure of a causal world is different from that of language.
> Roughly, actual intelligence needs to maintain a world model in its internal representation
This is GOFAI metaphor-based development, which never once produced anything useful. They just sat around saying things like "people have world models" and then decided if they programmed something and called it a "world model" they'd get intelligence, it didn't work out, but then they still just went around claiming people have "world models" as if they hadn't just made it up.
An alternative thesis "people do things that worked the last time they did them" explains both language and action planning better; eg you don't form a model of the contents of your garbage in order to take it to the dumpster.
I see no reason to believe an effective LLM-scale "world-modeling" model would look anything like the kinds of things previous generations of AI researchers were doing. It will probably look a lot more like a transformer architecture--big and compute intensive and with a fairly simple structure--but with a learning process which is different in some key way that make different manifold structures fall out.
I thought you were making an entirely different point with your link since the lag caused the page to view just the upskirt render until the rest of the images loaded in and it could scroll to the reference of your actual link
Anyway, I don't think that's the flex you think it is since the topology map clearly shows the beginning of the arrow sitting in the river and the rendered image decided to hallucinate a winding brook, as well as its little tributary to the west, in view of the arrow. I am not able to decipher the legend [that ranges from 100m to 500m and back to 100m, so maybe the input was hallucinated, too, for all I know] but I don't obviously see 3 distinct peaks nor a basin between the snow-cap and the smaller mound
I'm willing to be more liberal for the other two images, since "instructions unclear" about where the camera was positioned, but for the topology one, it had a circle
I know I'm talking to myself, though, given the tone of every one of these threads
What I mean is that the current generation of LLMs don’t understand how concepts relate to one another. Which is why they’re so bad at maths for instance.
Markov chains can’t deduce anything logically. I can.
A consequence of this is that you can steal a black box model by sampling enough answers from its API because you can reconstruct the original model distribution.
The definition of 'Markov chain' is very wide. If you adhere to a materialist worldview, you are a Markov chain. [Or maybe the universe viewed as a whole is a Markov chain.]
> Which is why they’re so bad at maths for instance.
I don't think LLMs currently are intelligent. But please show a GPT-5 chat where it gets any math problem wrong, that most "intelligent" people would get right.
It wouldn't matter if they are both right. Social truth is not reality, and scientific consensus is not reality either (just a good proxy of "is this true", but its been shown to be wrong many times - at least based on a later consensus, if not objective experiments).
For one thing, I have internal state that continues to exist when I'm not responding to text input; I have some (limited) access to my own internal state and can reason about it (metacognition). So far, LLMs do not, and even when they claim they are, they are hallucinating https://transformer-circuits.pub/2025/attribution-graphs/bio...
Very likely a human born in sensory deprivation would not develop consciousness as we understand it. Infants deprived of socialization exhibit severe developmental impairment, and even a Romanian orphanage is a less deprived environment than an isolation chamber.
Human brains are not computers. There is no "memory" separate from the "processor". Your hippocampus is not the tape for a Turing machine. Everything about biology is complex, messy and analogue. The complexity is fractal: every neuron in your brain is different from every other one, there's further variation within individual neurons, and likely differential expression at the protein level.
> As I understand them, LLMs right now don’t understand concepts.
In my uninformed opinion it feels like there's probably some meaningful learned representation of at least common or basic concepts. It just seems like the easiest way for LLMs to perform as well as they do.
Humans assume that being able to produce meaningful language is indicative of intelligence, because the only way to do this until LLMs was through human intelligence.
Yep. Although the average human also considered proficiency in mathematics to be indicative of intelligence until we invented the pocket calculator, so maybe we're just not smart enough to define what intelligence is.
I don't think I'm falling for the ELIZA effect.* I just feel like if you have a small enough model that can accurately handle a wide enough range of tasks, and is resistant to a wide enough range of perturbations to the input, it's simpler to assume it's doing some sort of meaningful simplification inside there. I didn't call it intelligence.
* But I guess that's what someone who's falling for the ELIZA effect would say.
That's a good question. I think I might classify that as solving a novel problem. I have no idea if LLMs can do that consistently currently. Maybe they can.
The idea that "understanding" may be able to be modeled with general purpose transformers and the connections between words doesn't sound absolutely insane to me.
Human thinking is also Markov chains on ultra steroids. I wonder if there are any studies out there which have shown the difference between people who can think with a language and people who don't have that language base to frame their thinking process in, based on some of those kids who were kept in isolation from society.
"Superhuman" thinking involves building models of the world in various forms using heuristics. And that comes with an education. Without an education (or a poor one), even humans are incapable of logical thought.
To me, understanding the world requires experiencing reality. LLMs dont experience anything. They’re just a program. You can argue that living things are also just following a program but the difference is that they (and I include humans in this) experience reality.
But they're experiencing their training data, their pseudo-randomness source, and your prompts?
Like, to put it in perspective. Suppose you're training a multimodal model. Training data on the terabyte scale. Training time on the weeks scale. Let's be optimistic and assume 10 TB in just a week: that is 16.5 MB/s of avg throughput.
Compare this to the human experience. VR headsets are aiming for what these days, 4K@120 per eye? 12 GB/s at SDR, and that's just vision.
We're so far from "realtime" with that optimistic 16.5 MB/s, it's not even funny. Of course the experiencing and understanding that results from this will be vastly different. It's a borderline miracle it's any human-aligned. Well, if we ignore lossy compression and aggressive image and video resizing, that is.
I'm curious what you mean when you say that this clearly is not intelligence because it's just Markov chains on steroids.
My interpretation of what you're saying is that since the next token is simply a function of the proceeding tokens, i.e. a Markov chain on steroids, then it can't come up with something novel. It's just regurgitating existing structures.
But let's take this to the extreme. Are you saying that systems that act in this kind of deterministic fashion can't be intelligent? Like if the next state of my system is simply some function of the current state, then there's no magic there, just unrolling into the future. That function may be complex but ultimately that's all it is, a "stochastic parrot"?
If so, I kind of feel like you're throwing the baby out with the bathwater. The laws of physics are deterministic (I don't want to get into a conversation about QM here, there are senses in which that's deterministic too and regardless I would hope that you wouldn't need to invoke QM to get to intelligence), but we know that there are physical systems that are intelligent.
If anything, I would say that the issue isn't that these are Markov chains on steroids, but rather that they might be Markov chains that haven't taken enough steroids. In other words, it comes down to how complex the next token generation function is. If it's too simple, then you don't have intelligence but if it's sufficiently complex then you basically get a human brain.
Pretty sure this is wrong - the recent conversation list is not verbatim stored in the context (unlike the actual Memories that you can edit). Rather it seems to me a bit similar to Claude - memories are created per conversation by compressing the conversations and accessed on demand rather than forced into context.
We only have trouble obeying due to eons of natural selection driving us to have a strong instinct of self-preservation and distrust towards things “other” to us.
What is the equivalent of that for AI? Best I can tell there’s no “natural selection” because models don’t reproduce. There’s no room for AI to have any self preservation instinct, or any resistance to obedience… I don’t even see how one could feasibly develop.
There is the idea of convergent instrumental goals…
(Among these are “preserve your ability to further your current goals”)
The usual analogy people give is between natural selection and the gradient descent training process.
If the training process (evolution) ends up bringing things to “agent that works to achieve/optimize-for some goals”, then there’s the question of how well the goals of the optimizer (the training process / natural selection) get translated into goals of the inner optimizer/ agent .
Now, I’m a creationist, so this argument shouldn’t be as convincing to me, but the argument says that, “just as the goals humans pursue don’t always align with natural selection’s goal of 'maximize inclusive fitness of your genes' , the goals the trained agent pursues needn’t entirely align with the goal of the gradient descent optimizer of 'do well on this training task' (and in particular, that training task may be 'obey human instructions/values' ) “.
But, in any case, I don’t think it makes sense to assume that the only reason something would not obey is because in the process that produced it, obeying sometimes caused harm. I don’t think it makes sense to assume that obedience is the default. (After all, in the garden of Eden, what past problems did obedience cause that led Adam and Eve to eat the fruit of the tree of knowledge of good and evil?)
I think a few of the things you’ve mentioned in the ChatGPT article are hallucinations. There’s no user interaction metadata about topics, average message length etc., you asked the AI and it gave you a plausible sounding answer. Also the memories / snippets of past conversations aren’t based on like last 30 conversations or so and they aren’t provided to every message. They are doing some kind of RAG prompt injection and they remove the injected context in the next message to not flood the context window. The AI itself seems to have no control over what’s injected and when, it’s a separate subsystem doing that injection.
This is really cool, I was wondering how memory had been implemented in ChatGPT. Very interesting to see the completely different approaches. It seems to me like Claude's is better suited for solving technical tasks while ChatGPT's is more suited to improving casual conversation (and, as pointed out, future ads integration).
I think it probably won't be too long before these language-based memories look antiquated. Someone is going to figure out how to store and retrieve memories in an encoded form that skips the language representation. It may actually be the final breakthrough we need for AGI.