But still - why is consciousness required? Because a model of the World could be held even without it, in my view.
E.g., I wouldn't think GPT-4 is conscious, but I'm pretty sure there's a representation of abstract World and relationships within it following the neurons and weights. Otherwise it wouldn't be able to do much of it, that it is.
Also I think model of the World is just that - which can be represented as relationships between neurons, symbolising that model of the World.
And I think you can have a complex and a perfect set of neurons and their connections to represent everything in the most efficient manner for that size of parameters (neurons and connections together). There probably is the perfect configuration, but it couldn't even be achieved using training or evolutionary methods.
I think most of our world model is actually a human model. Our social relationships are more important than we give credit for.
So there's an arms race. The more brains you have the more you can model your tribe to know how to help or succeed. AND the bigger everyone's brain is the harder they are to model simply.
In this model consciousness is the "self model" or "self-consciousness" that allows you to model others opinion of yourself by having such an opinion yourself. And adjust their opinion by providing a narrative about yourself which you first have to craft, .... nd on and on with higher levels of abstractions.
I’d argue that there is no default model. The model you have is just whatever you’ve spent time interacting with. If yours is mostly social then that perhaps plays a big role in your life. I would say mine is more heavily spatial and oriented around things and their interactions.
Supporting evidence for this are feral children who could never learn to integrate into society. They didn’t have built in models of social interaction.
feral children never develop what I would call the higher thinking. They still have mobility, spatial, etc. But "consciousness" is probably not something they really put together.
This is just my opinion. I think it took most of your brain to formulate that rebuttal, not concurrently, but as a deep and rich model of how readers might respond to a given argument, and weighing in a narrative sense your remembered experience vs what you "heard" me say in your head when you looked at these squiggly lines. Probably you even had a nascent image of me that was an amalgamation of your past experiences with internet idiots (fairly!) etc.
That's a lot, lot more than what a bug does when it finds its way past a complex maze, and probably something feral children lack.
I think all our higher level "reasoning" is a cooption of these systems.
There's a point in infant development where speech and social skills are developed, after which they cannot be. I'd go so far as to argue you'll never make a mathematician or physicist out of a feral child. Something critical to our higher level thinking is formed then. If missed, it cannot be recovered.
The flip side is I deeply believe if you can hold this conversation with me, then we both can be the best mathematicians there are, if only we took time to develop that language.
I like this model because consciousness just becomes another function, and it lets us explain some of the weirder modalities; for instance, being able to predict free-will choices way ahead of time in human subjects. Also, SMBC had a funny comic about this (funny comics always making science true!):
The reason I like this theory is that it provides the groundwork for and over eagar agency-modeller that explains a lot of human quirks like belief in gods, seeking root causes, modelling etc
I think some problems are simple enough that they can be dealt with "blindly", but some problems turned out to be tricky in special ways that evolution was more able to solve via consciousness than blind information processing. And from there, we find ourselves, with that new capability in hand, able to repurpose consciousness to newer and newer things. Then retrospectively it can look like consciousness wasn't "needed" for certain problems.
So I think even if you want to make the sense that consciousness solves a lot of problems it doesn't need to, it may have been a "real" solution to a "real" problem at some point in our history. And from that point on, it was no longer important whether it was the best solution.
I do think it's fair to say that lots of remarkably complex informational problems are solved in a p-zombie way, which is to say, with every outward appearance of intelligence (slime molds solving mazes, collective behaviors of ants). So I do think evolution or nature writ large "agrees" with you that consciousness isn't strictly necessary.
There's also the idea of the "hive mind" of ants and bees, so maybe quite literally our brains efficiently evolved as a meta-hive-mind internalized by our network of neurons.
That's an incredible way of thinking about it. I would say that is the most creditable way of dealing with the question of consciousness as it would apply to collective behavior, which is to say, it is a clear way of stating the difference between what those behaviors are, and how far removed they are from actual consciousness, without dismissing the "interestingness" of the behavior.
I would suspect though, that in our case it may have been something like centralized "nerve cluster" getting more and more complicated over time rather than anything that is in any sense a hive, although that is a very imaginative idea.
1) A huge stream on sensory data only some of which gets promoted to conscious awareness.
2) Some of that raw data and other conscious outputs are persisted into working, short, and long term memory.
3) And your consciousness works recursively using (2) as well as (1) as inputs.
All the stuff in GPT that gets called "memory" in machine learning seems much more like (1) and it lacks any ability to persist data outside its context window so we're still missing something.
Having a purely representative model of the world is less useful than having a sandbox for modeling choices and outcomes. Do I need to duck before entering that doorway?
That introspective analysis is consciousness. Humans have just improved the same mechanism allowing for more abstract analysis.
For instance, here I have GPT-4 with a system message: "You are an AI human agent in a video game performing actions.".
And I give it the following information as current environment:
Current environment:
1. Your goal is to get out of the current room.
2. You are moving in direction of North at 4km/h.
3. There's a door 3m in the North.
4. The door height is 150cm.
5. Your height is 180cm.
What is the sequence of actions that you plan to perform?
Its response:
---
1. Slow down as I approach the door to avoid collision.
2. Inspect the door for any lock or handle.
3. If a handle or a means to open the door is present, try to open the door.
4. If the door is locked, search for a key or any other opening mechanism within the immediate surrounding area.
5. If the door isn't lock or after opening it, crouch down slightly to accommodate my height which is more than the height of the door to avoid my head hitting the door frame.
6. Proceed through the door and stand tall again after passing through.
7. Continue moving North after exiting the room.
---
So here I did hint quite a bit by giving this specific environmental data, I could add more noise, and unnecessary information, but even then I think it can do it based on my experience.
I had temperature of 1 for the above, which I think why it made some odd word choices, but I also decided for example to change the info where the door is to "South", and this is how it responded:
1. Stop moving in the North direction.
2. Turn around 180 degrees to face the South direction.
3. Start moving towards the door in the South direction at a moderate pace.
4. Upon reaching the door, duck down slightly to accommodate the difference in height between myself and the door.
5. Proceed through the door to exit the room."
-------
So I guess, what I would like to see as an example. Assuming GPT-4 is not conscious, what is something that an average conscious human can do, thanks to presumed consciousness, that a GPT-4 couldn't do and why is consciousness required for that?
> what is something that an average conscious human can do, thanks to presumed consciousness, that a GPT-4 couldn’t do and why is consciousness required for that?
Consciousness isn’t required for anything. It’s useful for dealing with a complex and adverse environment not solving any specific problem.
That said, without testing I would assume if ask chat GPT-4 to change how quickly it responds it’s going to fail based on my understanding of their architecture. As in recite the pledge of of allegiance inserting 10 second pauses between each word. That said having provided a specific test it’s possible for that particular capability to have already been added to the UI layer via a pause token or something even if the model itself responds at the same rate.
If you used GPT as a brain you could provide the current time as an input for it, but otherwise yeah, GPT doesn't have time within it's input by default, but if you did:
Made the instruction to be: Say "Stop!" when 10 seconds are done. And then had a loop where
You give it current time in a loop, it would surely be able to do that.
But I'm not sure if or how that is exactly related to consciousness?
Time, the best way to think of it is probably a series of ticks anyway, and I assume in people's brains there is something to do that, so it would be reasonable to add a mechanism for GPT as well.
GPTs goal is to only act as the intelligent part of the brain based on the input.
Modify the system enough and it’ll eventually be conscious.
It’s not about a pause token, but the internal processes. You can have a long conversation on the subway with someone without forgetting you’re going home from work. Overflow it’s context window and GPT-4 has no recourse it just forgets. The difference is essentially prioritizing information, but LLM’s really don’t function like that it’s all about predicting the next token from a given context.
Give a future generation of AI systems internal working memory, a clock, and the ability to spend arbitrary time updating that internal memory and IMO that’s pretty close to consciousness. At least assuming it was all functional.
But it's not really difficult to inject this mechanism into the context window.
GPT-4 turbo latest version allows for 100k tokens or 75k words. The whole subway thing and more could easily be kept there, but what ever else can't can be designed in the prompt to always keep certain amount of tokens in context for different layers of memory, where you the more into the past you go the less details you have, but it's more like a Title of your most important learnings through out life, but at any given time GPT-4 can call the function to ask extra bit of content about it, if it seems relevant to the situation at hand.
So for example in each prompt context you would have:
1. Short description of what you have done each year in your life.
2. Key findings, goals, that you currently have.
3. The whole current day (or how much seems reasonable).
4. Past weeks in a bit more detail than the short description for a year.
So basically you could try to find 70k words to represent as much context and most important details (that are decided by GPT itself, what is most important).
I've been building an assistant for myself, that has such memory management system, it gets past N (like 40 messages) in full detail, then it will have summaries from before that time and in addition it will have messages and learnings in the past stored, and these will also be passed to the context depending on the query that matches it.
And if you want to compare it to human process of sleeping, it occasionally goes through the all the messages and "compresses" them, to extract most important findings and short summaries so they can be used in the next day's context.
So to me it's just basically giving it tools, and the other things like memory, longer term memory, inputs it currently doesn't get, are fine to be solved by other tools. I think that human brain also has different parts of brains working on different things, so it's similar in a sense.
Then once you have 70k spent on this historical context, you will run prompt in a loop allowing it to perform fn each time, like retrieve further info, or store some important fact, etc.
The real actual problem would be cost, because the costs would rack up quite quickly making looped 70k token requests.
USER
I'm responding to you roughly every second with current time, say "Stop!", after 10 seconds have passed. Respond only with silent when you do not wish to say anything.
Current Time: 13:04:11.21
Say "OK!" now if you have understood the challenge.
That’s almost but not actually waiting. Picking the nest token for a context window is something GPT-4 is quite good at. But the underlying approach has some real limitations which I discussed in response to your other comment.
Using 1 hour instead of 10 seconds and then list every Current Time incrementing by .01 seconds and you overflow the context window. The solution isn’t to have a larger context window, the solution is to have an internal representation independent of that context window.
The issue is you can’t train a system to use that internal memory by simply giving it a large corpus of text examples. The text lacks the internal representation associated with its creation.
That solves this specific issue, but it isn’t a general solution to the finite size of a context window. Ideally the system should be able to decide for itself what information it needs to pay attention to, but LLM’s simply don’t operate in that fashion.
But you can, e.g. you allow GPT to call functions to store embeddings that it can later retrieve for the memory. Meanwhile default context memory window is just recent info more detailed and historical info gets more and more summarized, possibly summarized by GPT itself.
I'm not convinced GPT can infer correct actions needed for novel scenarios for contexts that have not been directly experienced in training.
Consider:
I have never in my multiple decades of existence had a ball roll out in front of me while driving. But if I were driving by a playground and a ball rolled out in front of me from between parked cars, and the ball itself is not a hazard because it has already passed my lane, I can infer that I need to slow or stop because a child may rapidly follow that ball.
In your ducking scenario, the AI needs direct perception of a door. I assume it cannot imagine a need to duck without 1) direct perception of the doorframe, or 2) previous experience of ducking through a doorframe.
I'm not convinced that GPT or any AI can infer that context for handling novel edge cases at this point. It could just be a need for more training data to correlate different contexts, but I don't think it's there yet.
GPT has no sense, or care when it is wrong or right, such sense is only (arguably) driven by human through prompt interaction and throughout training of model, while humans and other animals able to update there's internal state just from single observation or interaction, and integrate future information with such single observation for very long time.
And other inputs from the environment. Then there were mechanisms which could also be neural networks that will transform this data into more digestible way for GPT and GPT was also in addition specifically trained to act based on this input.
Then it would run in cycles, where it gets this input, and it will provide output on how it plans to react to the data, maybe every 100ms.
It then could also have a storage it can use, where it can store data as part of the output to later retrieve it again.
So it would be a set of modules that is controlled and interpreted by GPT.
It could then do all of that above, no? And all of it should be just a matter of implementing. The only near time challenges may be certain types of inaccuracies and or producing tokens in some cases might take too long time to have fast reaction time.
So basically you'll try to run as frequent cycles as possible with the inputs mentioned above, other neural networks identifying the objects, in many different ways and all the context about the environment, unless a new version of GPT becomes completely multi-modal.
And you run those loops, then GPT gives output what it wishes to do, e.g. store some fact for later usage, move there, move here, etc. Or retrieve some information using embeddings then decide again, and short term memory would just be this context sized window, and if it needs more it just looks into its own memory for embeddings.
how system updates and maintains own model(s) when new information added in form of single/few observation and/or interactions, without ability to replay data, without catastrophic forgetting etc..., and importantly, how such system's model(s) grow in complexity while retaining stability and redundancies.
Analysis of what-if scenarios and simulating likely expected outcomes of proposed actions is a very useful ability, but IMHO that is not part of what we'd call 'consciousness'.
It’s a large fraction of what consciousness is actually doing, not the definition of consciousness.
Someone considering what to say is simulating tradeoffs.
In practice simulation isn’t limited to future possibilities. Trying to remember something you might recall vague sensory information like the tilt of your hips and try and create a scenario where that fits.
I would not automatically treat all the things that people do consciously as intrinsically linked with consciousness, unless there's some clear reasoning why consciousness is required to do that particular thing - and not in the sense that it's required because unconscious people can't do stuff because they're incapacitated.
Like, saying that simulating tradeoffs is what consciousness is actually doing seems to me equivalent to saying that smashing my fist on the table is something 'my consciousness' is doing, which is kind of technically true but not really relevant, as IMHO a non-human entity that's clearly without any consciousness could both smash something against the table and also simulate tradeoffs for various options, having a subjective experience isn't required for that.
Personally, I doubt that self-awareness can be achieved without some form of consciousness, and I feel that self-awareness is a key component of higher intelligence.
If intelligence and/or consciousness arise as emergent properties in the right sort of complex system, they will disappear from view in a low-level analysis of the causal processes occurring in that system.
Is there any way you could have a being like a human, who when asked would say they're not conscious? Is a definition of consciousness allowing that possible?
I'm not talking about whether they are or aren't, but surely all
intelligent beings would say and think they're conscious?
I mean you certainly could, in the sense that you could get it to tell you the sky is green, and that it's the spirit of Rodney Dangerfield.
To your point though, so far as I can tell it could not answer that question as a product by actively engaging conscious introspection to discover its own absence of consciousness, which I take to be your point.
So if its way of answering involves dipping into its own active "theater" of consciousness it likely could not, at least given our usual understanding.
However I do wonder if its something where that could actually be complicated to answer, and where, like illness, as we grow in understanding, it breaks out into all kinds of unexpected subclasses.
I can, for instance, know, in a sense, that I can't see UV rays. But I am not so sure if consciousness writ large could every be tested for, via conscious introspection, and come out in the negative, even if we allow for the idea that it will explode into all kinds of nuances and subclasses with future research.
A p-zombie is, by definition, physically identical to and physically indistinguishable from another individual who has conscious experiences, yet does not itself have those experiences. It's not clear to me that this makes more sense to physics, particularly when one considers the p-zombie and its conscious twin both being asked the same questions about their conscious experiences.
You don't have any experiences? Weird, I see a colored-in world, enjoy the aroma of fresh coffee, don't enjoy the wet cold in winter, don't like being awakened up from an engaging dream, and feel a mild sense of irritation by humans pretending not be conscious because of their materialistic commitments.
I have "experiences", they just don't really have the properties that a naive interpretation of sensory data would entail.
> mild sense of irritation by humans pretending not be conscious because of their materialistic commitments.
Funny, because I feel a mild sense of irritation by humans pretending that their flawed senses yields some kind of direct observation of fundamental reality.
Yet you observe them and don't put down their pretense to your flawed senses. Which suggests that something fundamental is impinging on your senses, however flawed your understanding of the underlying fundamentals may be.
> Yet you observe them and don't put down their pretense to your flawed senses.
I do actually. I think this internal sense of subjective awareness is just as if not more flawed than our other senses, and it has yielded all sorts of false conclusions because people consider it more reliable. I liken it to the illusion of multitasking on single CPU computers. I think Graziano is on the right track:
Having read the paper he's rapidly on his way to materialistic solipsism - which is fine as a circle of hell, but it doesn't have any bearing on reality.
Otherwise, he really isn't saying anything new from a philosophical perspective. Both Aristotle and St. Thomas long ago had a distinction between the act of sensing something and the act of naming (that is knowing) it. And noted that we can reflect on both of them.
As he says:
> Third, an attention schema cannot explain how a [non-empirically verifiable, non-material phenomena] emerges from the brain ... It explains how we believe, think, and claim to have such things, but it does not posit that we actually have [non-material phenomena] inside us.
Having limited himself to only material things, he is necessarily constrained to only talk about epiphenomena if there is a non-material formal cause for the material phenomena.
But empiricism is not all that is - there is no empirical proof for empiricism, it requires something outside of itself to support itself.
> I have "experiences", they just don't really have the properties that a naive interpretation of sensory data would entail.
You don't think your experiences have sensations?
> Funny, because I feel a mild sense of irritation by humans pretending that their flawed senses yields some kind of direct observation of fundamental reality.
I never said anything about fundamental reality. I don't understand saying we're literal p-zombies. You have to explain away your sensations, dreams, inner dialog etc., by appealing to some theoretical commitment.
I'm not claiming what consciousness is, only that we have it.
> You don't think your experiences have sensations?
I don't think sensations are what they appear to be either. Eliminative materialism is very misunderstood.
> I never said anything about fundamental reality
But you did, by claiming that we have consciousness. The only reason you think you have consciousness is an alleged direct perception of subjective qualities.
There are only two real paths to reconciliating this with some form of scientifically sound materialism: panspychism, where subjective qualities permeate all of reality and we are perceiving some of them, or concluding that our perceptions are flawed and untrustworthy and so the subjective qualities we think we're perceiving are not real/illusory (eliminative materialism). Given the demonstrable flaws in all other perceptions, the most parsimonious position seems clear.
Of course, you could try to resurrect idealism, but idealism is very problematic.
> I don't understand saying we're literal p-zombies.
P-zombies believe they have consciousness, say they have it, write whole screeds about how absurd it is to claim they don't have it, but they don't have the subjective qualities as we've defined them. Sounds like a bullseye to me.
> But you did, by claiming that we have consciousness. The only reason you think you have consciousness is an alleged direct perception of subjective qualities.
We don't perceive subjective qualities, subjective qualities constitute our perceptions, dreams, memories and other conscious mental states.
> There are only two real paths to reconciliating this with some form of scientifically sound materialism:
If you have a commitment to doing so. I don't particularly, since materialism is model of how the world is constituted, abstracted from our shared conscious experiences. A very useful one with scientific backing. But nature is more than what we think it is (map/territory distinction).
And there are other options you omitted such as identity theory, strong emergence, property dualism, neutral monism and cognitive closure. They all have their drawbacks, but so does every attempt to account for consciousness.
> P-zombies believe they have consciousness, say they have it, write whole screeds about how absurd it is to claim they don't have it, but they don't have the subjective qualities as we've defined them. Sounds like a bullseye to me.
Yes, but by definition, they lack the subjective qualities of our experiences. I believe Chalmers coined the term to demonstrate the problem with physicalism.
> We don't perceive subjective qualities, subjective qualities constitute our perceptions, dreams, memories and other conscious mental states
And I say the opposite, that our sensory perceptions, dreams, memories and other mental states drive processes that lead us to conclude that we've perceived subjective qualities in them.
> And there are other options you omitted such as identity theory, strong emergence, property dualism, neutral monism and cognitive closure.
To make a long story short: identity theory, neutral monism and panpsychism are basically the same on the property that everything has subjective qualities in some sense, strong emergence is a speculative concept at best that fundamentally amounts to dualism, dualism can itself be easily dismissed as any dualist theory will have a non-dualist equivalent that posits fewer entities and so dualism will never be preferred, and cognitive closure is not even a candidate as regardless of our ability to know consciousness, it still must logically fall into one of the categories I outlined.
>> P-zombies believe they have consciousness, say they have it...
> Yes, but by definition, they lack the subjective qualities of our experiences.
And, also by definition, each p-zombie has (at least in an accessible possible world) a physically identical, but conscious, doppelgänger. Speaking and writing are physical processes, so in any circumstance where the conscious doppelgänger writes about the reality of its consciousness, the physically-identical p_zombie will do the same.
In the case of the p-zombie, there was a causal process leading to the physical act of writing in which consciousness played no part - and, ex hypothesi, the same physical causality must have played out in the zombie's conscious doppelgänger. In general, whatever explains the zombie's observable characteristics is sufficient to explain the doppelgänger's observable characteristics, and that explanation cannot be dependent on consciousness. In particular, if one accepts the possibility of p-zombies, one must logically accept that no explanation of consciousness, regardless of how thorough and well-supported it seems to be, has any basis in any facts about consciousness.
A consistent belief in zombies seems to lead to epiphenomenalism, the notion that we have conscious experiences, but they play no part in what happens to our bodies. Frank Jackson, author of the well-known "Mary the neuroscientist" thought experiment, came to the same conclusion, which is why it appeared in a paper with the title "Epiphenomenal Qualia."
For more on zombies, see Dennett, "The Unimagined Preposterousness of Zombies."
Right, so one can reject the possibility of p-zombies. Consciousness plays a role in why we think we're conscious.
> In particular, if one accepts the possibility of p-zombies, one must logically accept that no explanation of consciousness, regardless of how thorough and well-supported it seems to be, has any basis in any facts about consciousness.
Although I disagree somewhat here, because logical isn't the same thing as physical. All you can say is there is no physical explanation of consciousness. But you could logically come up with one. For example, Chalmers has proposed an additional natural law connecting consciousness to information rich processes. Natural doesn't necessitate physical, if the universe contains more than physical stuff.
Or if physicalism is the wrong ontology of nature. I take the p-zombie and Mary Room arguments to be critiques of physicalism.
I feel you may be overlooking the significance of the final clause in the sentence you quoted. Sure, one could come up with some hand-wavy outline of a non-physical explanation of consciousness which does not clearly stand in contradiction with a belief in the possibility of zombies - panpsychists do it all the time! - but that final clause is about verification. If zombies are possible, then consciousness is not doing anything observable that is not attributable to just the physical processes of the physical body.
I suppose zombiephiles could simply posit some sort of non-physical observable, but then they would be just piling on further unsubstantiated claims for the sake of preserving the zombie-possibility doctrine. They could adopt epiphenomenalism, but that means accepting that the apparent causal effectiveness of qualia is illusory. They might adopt a solipsistic stance and say that their conscious experiences are the only things they can be sure about, but that comes with a hefty dose of motte-and-baileyism: you doubt the external world, but you're sure about zombies?
On Chalmers' "additional natural law connecting consciousness to information rich processes": Firstly, proposing something is cheap; turning it into a verifiable hypothesis and then following through is hard, and Chalmers et. al. seem to have no interest or intention of doing that, preferring instead to endlessly repeat their anti-physicalism arguments. Secondly, there is an attempt to redefine what's meant by 'physical'. The physical was originally defined by what's tangible to our senses, and as additional phenomena having a causal connection to the physical world were discovered, they were naturally subsumed into an expanded concept of it (gravity, for example, despite Newton's unease over doing so.) Chalmers is free to propose causes that lie outside of our current concept of the physical (and he's even free to look for them!) but to insist, without any justification, that they would be non-physical, is at least tendentious and probably question-begging. To settle the question, we need more facts, not more speculation.
I'm not sure what to make of your last sentence: the p-zombie and Mary Room arguments are indeed intended to be critiques of physicalism, but as you have rejected the possibility of zombies, I assume you do not take them both to be successful arguments. One thing about Mary's Room is that you can make essentially the same argument, but with Mary's prior knowledge including Chalmers' additional natural laws.
I'm not saying that there are no non-physical things. I am saying that certain philosophers are making seemingly tendentious arguments for declaring that consciousness is not a physical phenomenon. Once they get around to saying with sufficient specificity what they have in mind, then we will be able to see whether those claims are justified.
> And, also by definition, each p-zombie has (at least in an accessible possible world) a physically identical, but conscious, doppelgänger.
I agree that epiphenomenalism entails absurdities, therefore either all people in any possible world are p-zombies or p-zombies cannot exist in any world.
E.g., I wouldn't think GPT-4 is conscious, but I'm pretty sure there's a representation of abstract World and relationships within it following the neurons and weights. Otherwise it wouldn't be able to do much of it, that it is.
Also I think model of the World is just that - which can be represented as relationships between neurons, symbolising that model of the World.
And I think you can have a complex and a perfect set of neurons and their connections to represent everything in the most efficient manner for that size of parameters (neurons and connections together). There probably is the perfect configuration, but it couldn't even be achieved using training or evolutionary methods.
And none of it requires consciousness in my view.