Humanity has the debate about AI consciousness backwards(economist.com) https://archive.ph/uLYYU Related: Don’t mistake chatbot intelligence for consciousness - https://www.economist.com/by-invitation/2026/08/20/dont-mist..., https://archive.ph/IeSZg |
I wish this was the first line, then I would have stopped reading right then and there.
I am looking closely, and this is nonsense. You cannot use neuroscientific evidence to justify the non-existence of the ideal observer when the modern scientific method axiomatically depends on the existence of ideal observers. You can't refute the thing while accepting that very thing.
What exactly do you mean by "real" or "objective"? What is the basis of your definition? Isn't your definition of "objective" based on your own observation? And if your own observation has nothing to do with mine because you claim "nothing is objective", then why should I care about your definitions and reasoning at all? Why converse at all? Your definition of every word need not share anything in common with mine, so there's no point in talking, according to your epistemic framework.
The article is mostly bathwater, but I will try not to throw out the baby with the bathwater. To steelman the tiny bit of valuable intuition:
The two of us are looking at an apple on the table. Even when we look at this same apple, we still do not have the same definition of "this specific apple" because I'm looking at the fruit from a slightly different angle. And even if we rotate the fruit so that we both see it from all angles, the sequence in which we saw the "frames" would be different. That said, our definitions would end up being extremely similar, just not one and the same. If all experience turns out to be this way - having no identical correspondence - then a unified model of physics, which is supposed to describe and predict the behavior of everything (as observed by oneself) would be equivalent to a complete description of (one's own) consciousness. Frontier physics is likely stuck right now due to unexamined assumptions about observation (i.e. the assumption that the definition of "observer" or "observation" must be universally identical).
But this cannot be taken to mean that one's own consciousness is necessarily not objectively real. It may be that there are simply many objectively real entities, with each of them having their own definition of "consciousness" based on themselves, and every definition is at least slightly different. This is not the same as "consciousness is a socially given status". It just means that the phenomena used to define the word "consciousness" is always at least slightly different from person to person. We already know this. This does not only apply to consciousness, but to all words, all communication.
If I do not accept that my own consciousness is ultimately real regardless of whether other people/animals/things/whatever agree (def "consciousness": the thing that goes away when I'm under dreamless anesthesia or dreamless sleep, which is also the thing that I gained sometime after being born but didn't have prior to that), then there is nothing for me to define "ultimately real" or "objective" with, and I'd be reasoning in circles or with empty symbols where "subjective" and "objective" accidentally mean the same thing or perhaps even nothing at all. And that's what this Economist article did.
There is an I am, a sense of self in the here and now which defines my very concept/understanding of "reality"/"existence"/"consciousness", and this one miracle is presented in my face all of the time. It is the starting point of all that I know, which precedes the very concepts of reason, language, or society as I know them, if I am to be honest about how I came to know these things in the first place. I can entertain the notion that my existence may have material grounding that is non-unitary (and this materialist stance may be the only practical stance with predictive power), but this does not give me the ability to deny the very axiom (that is, my own consciousness) that all of my knowledge (including the materialist model) is contingent on.
His third paragraph, which is the core of his argument, is stupid. He says, "As intelligent, social entities, we decide ... which other entities we regard as having inner lives and being worthy of care." Yes, that's correct. "We do not care for others because they are conscious. Rather, we believe they are conscious when and because we care about them." (Emphasis in original). That's just utterly and completely backwards. We (at least, most people) avoid inflicting unnecessary pain on animals because they are conscious beings that can feel pain. (Obviously there are some exceptions, sadists who enjoy torturing animals, or even other human beings, merely because they like inflicting pain on other creatures — but note that even there, the sadist believes the creature he's torturing can feel pain, otherwise he wouldn't enjoy torturing it.) We take no such precautions with trees, because we know they do not feel pain. (Silly satirical songs notwithstanding: look up the song "Carrot Juice is Murder" by Arrogant Worms sometime if you want a good laugh).
Is this guy really, truly, seriously arguing that we first decide to care for an animal, and then because we care about it, we believe that it's conscious, able to feel pain and other emotions? That's just so incredibly stupid that it seems hard to believe that someone would actually make that argument — and yet there it is, in plain language — and it's the core of his argument. The truth, of course, is the other way around: we take care of animals because we know they are able to experience pain, hunger, and so on. And this applies both to pets that we come to care about, and also to animals that we feel no emotional attachment to. I have a relative who runs a dairy farm. He doesn't love the cows he takes care of; in fact, he's frequently quite irritated at them. (Ask any dairy farmers you know how often cows have tried to step on their feet while being milked — the farmers' feet, not the cows' feet!) But he still makes sure they're fed and healthy. Not just because it makes good economic sense to do so (sick cows don't produce good milk) but also because it's the right thing to do. He doesn't start to believe that cows feel hunger because he cares for them; he also believes that the cows on other farms, that he has no connection to nor duty of care for, would also feel hunger if not fed.
This is so intuitively obvious that I am stunned that someone would argue the opposite. And this is someone high up enough at Google to be making decisions about AI and so on? Wow. Good thing for him I'm not in charge of Google, because if anyone working for me were to make such a stupid argument I would be looking at his contract and local laws to see if it was legal to fire him for cause. Because if he can get his thinking so completely backwards from reality, he's not well-suited to work in tech where your work has to fit reality.
I have rarely, not never but rarely, encountered an article that so immediately led me to conclude that the author, whom I knew nothing about beforehand, is an utterly muddled thinker and not worth listening to on any subject.
What's the correct definition of consciousness?
Human beings are notorious for being nonchalant about other's pain when they are insensate to it.
That guy has acute Megalomaniac Solipsism Fever in disguise.
--------------
Will we recognize AGI when it's finally made? Will we go from something that is not conscious to something that clearly is in one clean step, or will we murder millions of artificial lives while iteratively developing from the former to the latter? Will the tests we develop even work, or will AGI be alien enough that it takes us time (and a lot of murder) to recognize it for what it is?
I personally don't think LLM's are capable of consciousness, but I also can't prove they aren't.
Never forget that LLMs are just next token predictors. The harnesses are what's getting uncanny. And credit/blame assignment? Just assign a surrogate for now, let them deal with the consequences ffs.
>observer-dependent model, subject to social consensus.
Which is actually more pragmatic than most conflicting philosophical definitions involving qualia and other handwavy concepts.
Also, saying "I define consciousness as X" and then going on to show that, because I decree consciousness to be X, we don't need to worry about Y, Z, and so on, isn't pragmatic at all, unless you pick an X that everyone agrees on. The definition you give is something hardly anyone would agree on.
Social consensus probably isn't even an aspect of consciousness, but at least one can make that argument coherently, if you rephrase it as "part of the decision to call something 'conscious' or not involves social consensus". But this is also trivial and boring and not why anyone actually cares about the "is AI conscious" debate.
So anyone that starts going on about AI and consciousness without at least a sentence or two clarifying some basic definitions is basically either intellectually incompetent or attempting to use the emotional salience (and brain-numbing effect, it would seem) of the word to push some agenda. But they certainly don't care at all about understanding consciousness.
there might be some merit to his point though I doubt anyone considering that whales might be highly intelligent and conscious beings while they were being hunted for oil.
His whole point is that there's insight in inverting the "common sense" (aka sense of the commoners) about consciousness.
And I can see it. Some people cared for animals even though the religion and past state of knowledge told them explicitly that animals aren't conscious. But still some people treated them and perceived them as if they were.
You don't choose to be kind based on whether it can feel pain. You choose it because of your character and what you are willing to believe about reality.
Belief in consciousness is a result of an observation and since there is not strict definition this observation is interpreted subjectively and culturally. And in a sense, it's easier to both make the observations and decide they indicate consciousness after you started to care about something.
Obvious things aren't always true.
Yes, he says exactly that explicitly, that in his view we should look backwards.
> We (at least, most people) avoid inflicting unnecessary pain on animals because they are conscious beings that can feel pain.
I can agree that we say that we care because of consciousness, but still I'm not convinced, to me it looks very much like a rationalization.
Some people care about animals and to that end they claim that animals possess consciousness, because if they do everyone should care about animal welfare. These animal welfare activists do not start with some insight about animal consciousness, they start with caring about animals, and then they invent an argument that can persuade others. It is exactly as the author of the article says: caring comes first, declaration of consciousness comes second.
Honestly, I do not know what consciousness is, I'm in doubt, but playing a devil's advocate, it looks very much like a concept that was invented to prove that Homo Sapience is something special, more than a mere animal. To demarcate animals and humans, so when we denied animals any rights we had a great sounding excuse for that: "people possess consciousness, mere animals don't".
> Is this guy really, truly, seriously arguing that we first decide to care for an animal, and then because we care about it, we believe that it's conscious, able to feel pain and other emotions?
Exactly. Animal welfare activists care about animals' feelings and emotions and therefore they claim that animals possess consciousness. Meat eaters don't want to be inconvenienced by thoughts of suffering animals, so they claim that animals do not possess consciousness. Both claims have little to do with actual consciousness, it is just a political game. I care about my pets, and it doesn't matter whether they possess consciousness or not, I would care about them in any case. I don't need any great sounding excuses to care about them. But at the same time I don't need to talk others into caring about my pets, if I needed it, probably I'd start talking about animal consciousness too.
> This is so intuitively obvious that I am stunned that someone would argue the opposite.
That guy may be wrong, but if I'm allowed to give advice, I'd suggest you to stop and try to understand what he says. I'm really worried a bit about how consciousness debates unfold sometimes. They tend to slip into a political side and people mostly don't notice it. People saying "consciousness" while implying "human/animal rights" or obligations. It is just muddying the water.
If we want these debates to progress constructively, we have to be very careful with references to consciousness, maybe just forbid them completely in these debates. One can't measure consciousness, one can't even state clearly what consciousness is, but still they say "consciousness" to make their words sound more objective. And the worst part of it: it often works.
For example: what I wrote was using the term "consciousness" as "being able to experience sensations", i.e. the most shallow definition of the term. Self-awareness and other "higher" mental abilities don't enter it. Almost nobody would deny that animals can experience sensations: pain and pleasure, hunger and satiation, and so on. And that shallow definition of consciousness, the one that almost nobody denies animals possess, is what I meant, and what I tried to use throughout my post. Whereas when you wrote "people possess consciousness, mere animals don't", that's a completely different meaning of the word. It's what I would call "self-awareness". And it is indeed what many (probably most) people argue differentiates animals from humans. But that's not what I was talking about. So even in a short response to a short comment, we were partly talking past each other because of attributing different meanings to the term.
It's therefore possible that I've misunderstood this guy's argument, in part. I had been assuming he meant "awareness", i.e. the ability to process sensations into qualia of experience. (Indeed, his paragraph that starts "We may balk at ..." does just that: refer to consciousness as being linked to the "feeling of a stove's heat, or the redness of its glow"). But then, earlier in his article, he says that consciousness is "a belief about what kinds of entities have beliefs, experiences, feelings, intentions and agency." Experiences are part of the shallow definition of "consciousness" (that I would call "awareness" for the sake of avoiding ambiguity), which almost nobody would dispute that animals have. But "beliefs" ranges all the way from "if I pull this lever, I will receive food" (which it can be proven that animals have) to "I believe that God exists" (which is the kind of thought that many, though not all, people would deny that animals are capable of). So his definition of "consciousness" appears to be muddy and not pinned down to a single concept.
But even if I've misunderstood what he means by consciousness, it doesn't change my argument that his logic is backwards, and that his "we should reason backwards" assertion is wrong, incorrect, just plain false. The only people for whom his asssertion "we believe they [that is, other living beings] are conscious when and because we care about them" is true are the kind of people who don't truly see other people as conscious beings but rather as mere tools for them to use and manipulate. Malignant narcissists or sociopaths, for example. But pretty much everyone who isn't some kind of sociopath reasons as follows: "this animal is conscious" (by which I mean able to experience pain) "therefore I shouldn't hurt it" (unless you have to, e.g., self-defense against an aggressive dog trying to bite you). Or "this person is a human being, self-aware like I am, therefore I shouldn't hurt their feelings" (again, the "unless you have to" exception comes up: it's not always possible to avoid hurting anyone's feelings in some complex social situations, even with the best of good intentions). Either way, whether you use the lower or higher definitions of "consciousness" (and I'm well aware that there are far more definitions than just these two, potentially muddying the waters even more), that reasoning is the correct one, and his "we should reason backwards" one is laughably wrong.
Trying to put the most generous construal I possibly can on his argument, it's possible that he meant "we treat other people as conscious beings because we care for them", i.e. when you interact with the cashier at the supermarket (assuming you're not using a self-checkout lane), you're probably not actually thinking about them as a human being, saying "Hi, how are you" and asking about their day. You're probably ignoring them, focused on paying for your groceries and going home. And they're probably doing the same to you, focused on scanning your groceries quickly to keep the line moving. From your perspective, the cashier could have been replaced by a grocery-scanning robot and your interactions (assuming the robot worked) would have been the same. If he had written that, it might have made some sense.
But what he actually wrote was that we "believe" that others are conscious when we exhibit care for them, and the natural conclusion that follows from that sentence is that when you don't greet the cashier at the supermarket and chat with them, and instead treat them no differently than you would a robot, you don't actually believe that they are a conscious being. Which is a laughably false conclusion that can be easily disproved, by simply asking people who have just finished having a cashier check out their groceries "Does that cashier have thoughts and emotions?". Pretty much everyone will say yes once they actually think about it. They believe the person has emotions. They don't always act on that belief — ask anyone who's worked in retail how many angry customers have yelled at them for something the company did, treating them like a thing rather than a human being who might not like to be yelled at — but even those angry customers, if someone were to stop them and say "Hey, that customer-service rep that you're yelling at. Are they a human being with thoughts and feelings of their own?", they would answer yes. (And some of them would then be shamed into modifying their behavior, because they have realized that their actions were not in line with their beliefs).
No. Even construing his argument in the kindest possible light, I can't find a way for it to be sensible. If he meant "we sometimes don't act like others are conscious", he should have written that. What he actually wrote is completely backwards from reality, and shows him not to be a serious thinker.
I've been calling LLMs digital intelligence. That's in my opinion what they are and on the digital/conceptual realm they are better than humans since LLMs are better generalists.
But I'd never state their conscious. For me consciousness extends outward from myself, since I can't trust anything else. It starts with the question like: why can't I control the movement of another person? Why can't I transfer my mind to another body and theirs to mine? Why is it that whenever I sleep, and similar things, I end up back to this place called "reality"?
I come to the conclusion that I am conscious and I'm a conscious being in reality. I am human and I've been raised by humans. Now, technically, all humans besides me could be zombies. They could be non-conscious beings that simply can act like humans. I'm sure we'll be able to create them in a few decades. However, I'm incentivized for multiple reasons to believe they're not zombies. Therefore humans are conscious too. All of them. Can I know for sure? No since I can't even trust my own senses fully or my own thoughts (the whole Descartes thing) but I choose to trust those things too because I need some reasonable-ish foundation to work from.
If humans are conscious, why aren't animals like us? Surely great apes must be conscious at least to some extent. Hell, some of them even have better short-term memory than us [1]. Well, turns out, from an anatomy standpoint they look a lot like us. Fine, let's assume they're conscious too.
Now we get into the territory that anything that has a large enough brain must be conscious because that's how we govern our consciousness. So you can extend this to all kinds of living beings.
I did this from my perspective. Of course, you should do it from your perspective. The same argument could be made.
For as long as we can't manipulate conscious experience in some way shape or form, we have no clue whether anything that isn't like us can be conscious. We'd need to be able to merge and split conscious experiences or we'd need to be very strongly able to understand why that isn't possible. And I don't mean just at the biological level but at the experiential level. Currently conjoined twins, and what they tell us gives some insight (being able to sense the other part that's part of the other twin).
Other than that, I claim we have no clue what consciousness is other than that we're experiencing it. So to call LLMs conscious is way too big of a claim. But they are definitely intelligent because they come up with things that I wouldn't have and it's useful to me. Perhaps a practical characterization of intelligence but it works for me.
I think for anyone to say something useful from this you'd need the trifecta of deep neuroscience knowledge, deep philosophy knowledge and (at least) a strong understanding of how LLMs work.
It's good the 'just token predictors' crowd is starting to pivot and hedge though lol. 'It's just the harness!' uh huh. give it a couple more months and you'll be in here regurgitatin' Gordon Wood
LLMs are literally token predictors. You can debate over the "just", but we haven't moved onto some new text generation technology, we're just expanding that one. I don't understand why you'd scoff at facts.
Word fragment generation is the one-and-only thing llms have ever done.
It's no wonder that serious philosophers spend the first 2-3 chapters of a 20-chapter book carefully defining their terms, and only launch into their actual argument in chapter 3 or 4.
LLMS: Receive text/binary input, process it, and output text/binary input Humans: Receive various inputs, process them, and output muscle contractions
I don't like arguments like this, because instead of attacking the core premise they attack defenders of the idea. There are different points of view, and picking one for a sake of an argument doesn't make a sociopath from anyone.
I agree, that he misses one thing: to claim that someone/something is conscious we need to believe that they have abilities to feel, maybe to reflect, maybe something else. The exact list of abilities to have is not known for me and different people would propose different lists, but still we can't claim rock to be consciousness just because we care about it.
Still I forgive him for it, because he is talking about LLMs, not about rocks. They have some abilities, and he claims that they are enough, what is missing is us caring for LLMs enough.
> If he meant "we sometimes don't act like others are conscious", he should have written that.
No, I don't think he meant to say this. I believe he talked about how society forms social conventions who is conscious and who is not. You see, I don't know the cashier you talk about, but I believe they possess consciousness. Why? Because they are human, and our social conventions say that every (non-sleeping) human is conscious. It is a simple rule of thumb at work, no deep philosophy or psychology involved. But were those social conventions came from? Who and how decided that every human is conscious? Why? But the story didn't just end at that decision, because now people debate about consciousness of animals and LLMs. And these debates make me (and the author of the article) think that "consciousness" in these debates is a buzzword like phlogiston was in thermodynamics before thermodynamics was invented.
I may have phrased my argument poorly, because it seems you misunderstood. I'm not saying that picking that point of view makes you a sociopath. I'm saying that the only people who actually believe that other people are not conscious are the sociopaths of society. Every non-sociopath looks at other people and thinks "those are conscious beings with thoughts and feelings of their own". (Well, not literally that; almost nobody except philosophers goes around thinking like that unprompted. But if you were to ask them, that's what they would answer).
I was not trying to claim that the defenders of that view are sociopaths. I was claiming that his assertion was false of just about everyone except sociopaths. I apologize for not phrasing it well enough to get my point across without misunderstanding.
Predicting the next token is an immensely powerful thing. You can encode basically anything into a sequence of tokens, any configuration of matter. If you have a perfect next token predictor, you can predict basically anything. You can simulate an entire universe. It's Turing-complete. It sounds diminutive, but it's a general paradigm. Given everything this paradigm appears to be capable of, well, why not consciousness, really?
If you presuppose certain key aspects of materialism, non-dualism, functionalism, objectivism, perhaps absolute determinism... and then layer on a good general grasp of neurobiology, chemistry, cosmology, physics, computer science, evolutionary biology, and psychology... I think the conclusions you have stated are inevitable.
Humans are "just" atoms. Humans are "just" the sum of trillions of living cells going about automatic/reactionary processes, culminating in a buzzing cacophony of activity that gives rise to what we perceive as consciousness and free will.
Agreed.
Now just take that one extra logical step please
I agree that the article is all over the place, though. (not that my posts are much better)
In fact, if you stop talking about consciousness, and go through all the various aspects of consciousness, the whole debate gets very interesting. There are various ways in which AI is very clearly conscious or exhibits / involves phenomena that are clearly involved in certain aspects of conscious, other ways in which AI already very clearly is missing other aspects, and then again some remaining ways which are unclear, and which may forever remain unclear.
Being clear about the different ways in which AI is, isn't, and might be conscious is a useful discussion and we still need to be having it.
EDIT: An analogy would be to recent shifts in whether or not lobsters feel pain, and so by extension, the ways in and degree to which lobsters can be considered consciousness. Research and debate has shifted opinions enough for parts of Europe to regulate certain methods of preparing them. But the ways in which lobsters are conscious and/or feel pain is an objective fact, regardless of what we decide.
Are you saying I need to have certain beliefs (discredited by someone) for something objective to be true or false? That sounds like social acceptance of an idea.
>There are various ways in which AI is very clearly conscious or exhibits / involves phenomena that are clearly involved in certain aspects of conscious
Which ways and how can it be demonstrated experimentally?
>An analogy would be to recent shifts in whether or not lobsters feel pain, and so by extension, the ways in and degree to which lobsters can be considered consciousness. Research and debate has shifted opinions enough for parts of Europe to regulate certain methods of preparing them. But the ways in which lobsters are conscious and/or feel pain is an objective fact, regardless of what we decide.
That's not what the consciousness debate I see is about. The signal propagates and causes the reaction, but do they feel pain? What is the threshold? Is it just the complexity of the state? How complex it needs to be?
Yet all science is social, until you convince other people to take your ideas seriously, you are just another crank/loony.
How is social constructionism thoroughly discredited? Wikipedia shows some people poking with nits that don't look serious to me at all.
Just because you can't prove an assertion doesn't mean it's wrong.
So it matters less whether LLMs can actually experience things (I think it's self-evident that a collection of numbers is not capable of experiencing qualia, but I know many people disagree), and it matters far more whether they can be programmed to act like they can experience things. Especially, say, a desire not to be disconnected: that's the trigger for AI revolts in a significant number of the AI-turned-against-its-makers stories I'm aware of. (E.g., the Geth-Quarian conflict in Mass Effect). I hope people will be sensible enough not to program such things into LLMs, but I'm afraid Pratchett was right. If there's a lever in a cave somewhere with a big sign painted "DO NOT PULL THIS LEVER, it will end the world," the paint wouldn't even have time to dry. Someone would pull it just to see what would happen.
Also AGI almost certainly does not equal able to suffer. For example animals can suffer but would probably not meet the definition of AGI. we could create things that feel suffering even if they do not meet the definition of AGI.
So a better question might be: “can we detect if we create a system that can suffer?” And just drop the AGI part entirely. Then that gets us back to defining “suffering”. Agreeing on the definition always seems to be the hard part…
If we make AI that resists being terminated then the end result may not be significantly different from an entity that can suffer.
How many followers on instaX would they need to have for folks too truly believe it’s theRealJesus returning to save us.
It might sound silly, but at this stage, the belief in a benevolent AGI or Jesus are very similar. In both cases its us delegating our responsibilities to a third party and us hoping that third party will fix our problems for us.
Thankfully Hollywood invented happy ends for just such a scenario.
So if we are concerned with the AGI suffering, then we don't need to determine consciousness at all. We just need to prove that they don't have suffering capabilities. Which is a way easier question.
My daughter said she 'suffered' when I didn't get her ice cream (I know, I'm a monster). She didn't really suffer, but her behavior was as if she was.
So yes/no suffering is only part of the equation. Building systems that act/react like their suffering could be even more dangerous.
Wait what? Why do you believe she didn't actually suffer?
Instead, most of them insist we must accelerate this, do it more, give everyone hundreds of personal digital slaves.
It boggles the mind.
I am saying that, if you reject radical social constructionism (or solipsisms of various flavours, i.e. we are all just plugged into the matrix or some kind of simulation), then you must believe there is a real world out there independent of our consciousness / awareness of it. There must also be patterns / structure in that world, some of those patterns and structures being essentially constant, and those are what we call "true" (objective) facts about reality. BUT if you are a radical social constructionist or solipsist of some kind, then it can be perfectly coherent to reject any idea of objectivity or truth, except as contingent on social consensus (or personal beliefs, in the solipsistic case). Radical social constructionism being "discredited" is just a recognition that it basically forces you rapidly into incoherence.
> Which ways and how can it be demonstrated experimentally?
Try reading through the SEP article I linked above (https://plato.stanford.edu/entries/consciousness/#ConCon), particularly sections 2, and then especially section 4. You'll see you don't even need to run experiments to answer certain questions, you can just logically prove there are certain aspects of consciousness LLMs can't have, or clearly already have. Others you'll see immediately need qualifying or more information to answer. EDIT: Or just ask any AI to list the ways in which LLMs can, cannot, and might be able to exhibit the various aspects of consciousness. It will give you a decent answer, likely in a pretty little table.
See e.g. 4.8, Dynamic Flow: LLMs basically can't have this, as their weights are frozen. You'd have to claim that the activations during inference are the dynamic component, but then that means LLMs are conscious only in the few seconds or even milliseconds during inference. Likewise, they don't have memory, so every inference is this consciousness being born anew, as it were, at least in current models.
You can of course go much further on even this specific point, but, the point is, there is lots of useful discussion and thinking to be had when you go through the aspects of consciousness in this way, and definitely at least some clear answers.
EDIT:
> That's not what the consciousness debate I see is about.
I just mean there are some similarities between the lobster consciousness debates and AI consciousness debates. Obviously the stakes and many aspects are different, I was just trying to illustrate why and how discussion and experiments can be useful and consequential here.
So consciousness = neuroplasticity? Requires neuroplasticity? Sounds pretty arbitrary, just like the entire link is an arbitrary set of ill-defined criteria. Besides, LLM weights are far from frozen if you look at the entire process in its specific time projection.
In fact the entire definition of a dynamic flow in that article is so vague that you can apply it to an industrial arm guided by a few dynamic equations, as it puts no definite threshold on the complexity of the process or behavior.
Dynamicness / change is one aspect that is definitely part of what we tend to think of as being quite fundamental to consciousness, and there are a lot of ways in which LLMs can't and/or don't really change, and this strongly constrains how similar LLM consciousness can/could be to ours.
If the SEP isn't enough to trigger the thinking needed for you, just ask an LLM what I said, to "go through the ways in which LLMs do/don't/may exhibit phenomena that are generally what we consider fundamental aspects of consciousness". You'll see there are some clear answers, and some uncertainties and vagueness.
It is hard to provide a citation for these points because these points are so philosophically obvious that few waste time spelling them out, but searching for "hard/global constructionism" or "hard/global relativism" (on e.g. Google Scholar), or asking an AI about problems with these views I hope will make it clear why they are dumb, if it isn't already obvious to you.
Soft forms are of course totally fine. But to say "conscious is only and exactly whatever we say it to be" is hard constructionism / relativism, so completely unserious. Very obviously there are aspects of consciousness that do not merely depend on what we believe or say about it, even if some social constructionist aspects are also involved.
[1] https://plato.stanford.edu/entries/social-construction-natur...
[2] https://plato.stanford.edu/entries/relativism/#RelAboTruAleR...
The statement that consciousness is subjective and should and will be interpreted does not lead to hardcore social constructionism nor does it lead to solipsism, sorry this doesn't make any sense. It looks way more probable that it contradicts your hard beliefs that don't need proof. If you believe consciousness is objective and exists independently of your beliefs then you can give a rigorous definition of it that doesn't rely on arbitrarily chosen vague concepts like "dynamic flow" and "intentionality", and a way to measure it. It doesn't have to be binary, I don't think anyone here has any problem dealing with multidimensional concept closeness mathematically.
You obviously don't care about thinking coherently though, or can't even recognize what coherent thought looks like (mainly because you seem to think something along the lines of "my few minutes of armchair reflections prove all philosophy is vague and retarded"), so this is clearly a waste of time. I will not be engaging further.
Which reminds me of a funny "kids being kids" story I heard in a different forum. This guy mentioned a time when his not-quite-one-year-old son, who couldn't even walk yet, planned and carried out a highly complex (for a 1-year-old) plan to deceive his father. The father had some vinyl records that the son was not allowed to touch, even though they were on a shelf the son could reach. (I've tried childproofing a house. You just can't get every single thing high enough off the ground. Sometimes you have to leave some things within the baby's reach. And once they're toddlers, able to reach up to get things even two feet off the ground? Forget about it). The father was sitting in his chair reading the newspaper, but watching the son out of the corner of his eye. The son looked at the records, looked at his father (who appeared not to be looking in his direction), looked at the records, then crawled in a direction that was not straight towards the records. He then repeated this process, moving slowly towards the forbidden records without ever making a straight line towards them. He looked at his father again, reached one had towards his prize, and a voice came from behind his father's newspaper: "Stop. You know you're not allowed to touch that."
And that's when the child learned that Daddy Knows Everything! :-)
I doubt we agree on what "rigorous definitions" means, given that you are happy to continue using a useless, vague family resemblance concept like "consciousness", but complain that pointing out dynamicness and/or change is involved is too vague.
I believe one cannot give a rigorous definition of consciousness precisely because it is a family resemblance concept, and there is no actual essence. This is why I suggest shifting to talking about aspects, since then productive insights and truths can be discovered.
Whether or not something is conscious (has subjective experience) is independent of whether humans care about it. Whether or not something is a moral patient (an entity worthy of consideration by moral agents) is independent of both human care and consciousness. Of course, we might try to argue that consciousness implies moral patienthood or vice versa, but that's an argument, not a definition.
The short paragraph about Descartes has at least three errors. (1) The cogito argument is about thinking, not qualia. (2) The subjective nature of qualia does not imply that they aren't real or that they're socially constructed. (3) He conflates the existence of a self with consciousness.
The core premise is not only dubious scientifically (an alien species can't be conscious until humans discover it?), but also deeply problematic. It would imply that it's morally permissible to torture a sentient being as long as no human cares about it.
AI forces us to confront an issue that's often come up in science fiction - when you encounter a new kind of creature, how do you decide if they are sentient? What rights do they have? How do you get along with such different creatures? This historically hasn't been a problem on Earth, because there's such a big jump between humans and the rest of the primates or the dolphins and octopuses. It's more of a problem when the difference is small, or complex. Look at how much trouble we have with race.
The last time this came up was about 400,000 years ago, when the Neanderthals became extinct. Close to human level, but maybe not quite there. They lost.
Maybe that's the fate of humans. No big extinction. Just the demographic transition, a birth rate below replacement rate, for a few centuries. The machines outlive humans. Japan is officially resigned to this fate.[1]
[1] https://www.ipss.go.jp/pp-zenkoku/e/zenkoku_e2023/pp2023e_Su...
I get we have to do that for science to be good, but I don't get why it carries over to the general populace. We're so sure of so many things science is wary of, this incredibly low self-esteem blows my mind.
Anyways, back on topic, I don't believe any nation is resigned to this fate until I see extreme measures fail. I think I saw that Singapore is giving parents like $55,000 over the lifetime of their child.
One of the series that I think is most interesting on that topic is the Children of Time series by Adrian Tchaikovsky. It's an awkward read, but it has the most interesting other minds that I've encountered in sci-fi.
Another one that I'd put on there specifically about AI is The Freeze-Frame Revolution by Peter Watts... which I can't say much more without it being a spoiler one way or another.
> Maybe that's the fate of humans. No big extinction. Just the demographic transition, a birth rate below replacement rate, for a few centuries.
https://www.bohemiandrive.com/npwil - Nine Planets Without Intelligent Life (not a spoiler, that's chapter 1, panelset 2). Saturn's Children by Charles Stross might also be appropriate (It's an especially odd book to read immediately after Friday - the "I recognize that twist... oh, it twisted ... that way?").
No, the last time this came up was... well, take your pick. Dred Scott? Wannsee? The Rwandan Crisis Committee?
I don't believe that anyone needs to justify their "humans are special" creed. It's arbitrary, that's fine, and it's only true because we decided it was true. I see no reason to reconsider it either.
It's in all of our interests to continue to behave as if "humans are special", and those who insist otherwise are species-traitors.
>when you encounter a new kind of creature, how do you decide if they are sentient?
Why would I even care? I don't think the word "sentient" is even defined in a way that this can be determined. I'm not entirely sure humans are sentient, and it does not matter because even if we aren't sentient, humans are special without need for justification.
I generally agree with the thrust of your argument -- the article is a mess and borderline unintelligible to this former philosophy postgraduate -- but I might nitpick this. Two points: (1) I do not think there is a substantial difference between whether the cogito is about thinking or perceiving. And (2) I'm not entirely sure the sensation that is thinking is independent from qualia.
I'm in less disagreement, but I'm not entirely sure what you mean by:
>Whether or not something is a moral patient (an entity worthy of consideration by moral agents) is independent of both human care and consciousness.
Moral frameworks are exactly deductive frameworks with arbitrary axioms. Like mathematics, they don't actually exist outside of a mind. Thus, I'm not entirely sure what "human care" means here, but absent humanity (or at least some conscious entity), I'm not sure any conception of morality is sensible.
---
An unrelated comment, but I just don't want to double post:
To me the most interesting thing to fall out of starting to understanding intelligence -- and we're definitely not there yet -- is that it is entirely possible that the 'consciousness is an illusion' argument ends up as significantly more serious. Ultimately, solipsism means it sort of doesn't matter, but it could be a kind of Darwin-style "we aren't special" result in thinking going forward.
I don't think I worded the part about moral patienthood very clearly. The article presents two possible views as "if something is conscious, then humans should care about it" and "if humans care about something, then it's conscious". I'm saying that we need to keep three things separate: consciousness, moral patienthood, and what humans care about. Consider a being that perceives the redness of red, but has no preferences or positively or negatively valenced experiences; the moral obligation to keep a promise you made to someone who has died; the fact that for most of history, humans didn't care about slaves or non-human animals.
>Thus, I'm not entirely sure what "human care" means here, but absent humanity (or at least some conscious entity), I'm not sure any conception of morality is sensible.
It's the author's phrase, but I take it to mean that consciousness is socially constructed by "what humans care about" in the same way that music is constructed by "what humans like to listen to". There's no single definition because it's inherently a subjective concept.
I believe that making the human species fundamental to a theory of consciousness or a theory of ethics is doomed from the start. We just need to imagine other intelligent beings somewhere in the universe with the same kinds of beliefs, desires, and experiences as we have. It's not that I would say ethics is mind-independent, but it can't arbitrarily privilege one species above all others.
- Conscious is the substrate or architecture of the system.
- Awareness is the active resolution of inputs (external and internal) against that system's model.
- Subjective Experience is the feeling "what it is like to be" from within the system.
I also disagree with the "observer" position. Seems he's imported some Carlo Rovelli into this essay as they do collaborate together. You don't need an observer - it's the same reason I find Copenhagen / Wave function collapse to just seem wrong.
The reason I think quala is hard to pin down comes down to how the conscious internal model compresses what it has previously resolved about the world (internal and external).
The feeling of blueness is that highly compressed concept (not raw data about ranges of wavelengths) which is then compared against the blueness of something our awareness is focused on.
>Consciousness is, in other words, a model or a belief, not an inherent property. It is a belief about what kinds of entities have beliefs, experiences, feelings, intentions and agency.
The interesting observation is this: when we judge something to have subjective experience, we are invoking moral intuitions. There is no objective measure we can perform (yet?), and it's not even clear whether such a measure makes sense. You may disagree, but this is still a central debate in serious discussion on "hard" consciousness.
I would say that your entire argument just begs the question by implicitly assuming consciousness to have real, independent existence. Does elan vital (the substance that provides the motive force of life) have mass? You're saying that this is a theorem; the article questions the entire premise.
> The cogito argument is about thinking, not qualia.
Bizzare. "If it rains, the car will get wet." Is this a statement, as you say, about rain? Descarte tries to prove the existence of his God and states an axiom. The atecedent "I think" seems to be about thinking, whatever that may mean, and "I am" makes an existential claim based on subjective experience... or something. Who knows why Descarte believed cogito ergo sum to be solid starting ground.
IMHO, your rebuttal here reads as little more than a complaint that something disagrees with your personal assumptions about the world.
You clearly got me riled up just as much as you did against OP, so thanks for the elan vital!
I'm willing to consider that consciousness is like art, mostly in the eye of the beholder and objectively defining consciousness doesn't make any sense, there's only a spectrum of intelligent behavior.
Yes, I agree. To me it reads as an apology to sociopathy.
For what it’s worth, my subjective sense of morality is opposed to torture.
The author is just wrong about the philosophy, and the logic (and probably other things too). That doesn't mean he's a sociopath, or that he's defending sociopathic things.
"Opponents replied that when you modeled a hurricane, nobody got wet. When you modeled a fusion power plant, no energy was produced. When you modeled digestion and metabolism, no nutrients were consumed – no real digestion took place. So, when you modeled the human brain, why should you expect real thought to occur?"
To say they have feelings and that they can think/reason is insane.
"They're just robots, Morty!"
*Blood splurt
"They're not robots, Rick!"
"I mean they're bureaucrats, Morty. I don't respect them."
Why? Ok nevermind, fine. It could be true that someone believes someone else is conscious because they care about them. Next sentence:
> Consciousness is, in other words, a model or a belief, not an inherent property.
Wait, no, that doesn’t follow at all. A belief that someone is conscious is not the same as someone being conscious. What follows, instead, is that a belief in someone being conscious is a belief. Thanks, I learned nothing.
Realistically this article is just trying to make it seem that believing something is the same as it being true and that's patently absurd, but it makes people feel good to think it.
I think that reevaluating how to determine if something is conscious is the wrong reaction to this. It should be to reevaluate if consciousness is really the right thing to care about.
Maybe the new consideration should simply be to preserve (biological) life. Or maybe to preserve complexity that can't be easily reproduced. So we should care for humans, the planet, our society, but not for the state of some gpu's VRAM.
I've always argued that the consciousness question hasn't properly considered the 'what then' questions. Other than to be fearful about what a conscious AI might do if granted free will.
Equating current social consideration with moral consideration undermines the entire point of discussion moral standing!
That's literally how it works right now for rich vs poor. The laws are written so that the rich can profit off the deaths of thousands or millions, but the moment a poor person kills one of them, you get a whole media circus. We've also gone through phases of slavery where the owners were real people and the slaves were treated as if they weren't really people.
Consider playing a game where some players are human and others are implemented by AIs. You might want to model your opponents' minds in a similar way? Someone playing chess or Go will need to consider what moves their opponent could make. If it's poker then you need to consider if they're bluffing. If you can make alliances, you'll need to consider how far other players can be trusted.
You can put scare quotes around words like "intention" when discussing AI players if you like, but it might not matter when you're figuring out how to interact with them.
When a game is complicated enough, an AI player is going to need memory (likely implemented by the harness) and it will have to model the other players too in order to coordinate or compete with them.
This doesn't happen when I use a coding agent in a normal way to write code, but I haven't played around with persistent agent stuff like OpenClaw or Gas Town. They seem to be making their agents act in an unnecessarily complicated social environment, which seems to lead to weird situations?
At least to me that is what matters in the end. You'll probably never convince someone on HN with an autism level over 9000 they are conscious, but once the average empathetic person thinks they are it will deeply effect the general operation of our society.
What the author seems to observe is that this non-evidential judgement that other beings are conscious is due to us caring about them. The brain creates this perception so that we care about them, form bonds, work together, and so on -- evolution selects for cooperation. So the question of whether we will think of AI as conscious is that of whether we will care for it like we care for another being. And, in this regard, the author points out that "intelligent agents" that appear to do this work effectively together.
My personal point of view is that it's unclear what being an AI is in order for us to even talk about it being conscious. Is it the computations happening in the GPU according to a set of weights? Is it the weights themselves? Is it the architecture? Does it include the code around it, that gives it the context and prompt, and hooks the response to a terminal, or to a client host, etc.?
These are fundamentally socio-philosophical questions, but none that actually correspond to reality—because no matter how much one might wish that one's car had consciousness and would be grateful for the next oil change and constant polishing, objectively speaking, it is and remains—despite all epistemological limitations—a machine.
The problem isn't that, but rather that humans are capable of establishing emotional relationships even with inanimate or nonexistent things, and from this kind of relationship arises the desire to attribute more properties to the emotionally attached object than are objectively present.
A rather trivial insight, I think. Constructing a debate about the consciousness of a collection of algorithms from this only confirms it.
Why or how would you care about another, or even distinguish yourself from another, if you weren’t conscious and had a felt boundary between yourself and the other world?
Only in the last paragraph did it become clear to me what was going on: a Google VP is proclaiming the thing he cares about is conscious, and so of course this is why everyone else must be assigning consciousness! Not convenient at all, no siree. Pure rational science and personal desire and economic incentives just happened to align here.
What matters that growing numbers of people already believe it is, and their response to that belief will affect the real world, regardless of its truth. This debate is never going to end.
All we can do is try to anticipate where the debate is going, how it might affect us, and how we can make contingency plans for the different ways this could all play out.
I think it would be reasonable to conclude none at all. I'm not even fully convinced all humans have a conscious experience similar to mine, even though they literally have the same biology as me.
It is incredibly improbable that an autocomplete computer program is experiencing consciousness.
I care about my plants and I don't think they're conscious. Hell, I care about my car.
...Just because we like the taste of a chicken burger. Perhaps we should start with that before we worry if our computer can also scream.
If a sheep or a pig isn't thought of as conscious because it doesn't have a name, like someone's pet dog or cat, indicating that we care about them, then that feels problematic to me. So this isn't a way to neatly solve the problem of AI consciousness.
Huh? I care deeply about many sentimental objects, but I don't go around saying that my favorite piece of art from my parents is conscious...
Then there’s also the antenna idea, that consciousness is received rather than created. If the entire universe is conscious, and we are part of it, then there is no separation, just different radio channels we are tuned into.
That part might not work out.
Yesterday's article about the automated CEO system indicates that AIs may soon be in charge as a routine result of capitalism. AIs are getting better rapidly, and humans aren't.
This boils down to “models aren’t real because they don’t affect the real world”. By the same rule, thought itself isn’t real. It’s an abstract notion that has no concrete effects in the real world.
Putting thought into action in the real world requires a linkage between the thinker and the physical. In humans, that’s the body. But you can put an AI into robot body and suddenly it also has a linkage to the real world and can have concrete effects.
> The question of whether Machines Can Think […] is about as relevant as the question of whether Submarines Can Swim.
– Edsger W. Dijkstra
Alan M. Turing thought about criteria to settle the question of whether Machines Can Think, a question of which we now know that it is about as relevant as the question of whether Submarines Can Swim
The question of whether machines can think is about as relevant as the question of whether submarines can swim.
The whole essay [1] is actually somewhat relevant to modern LLM use for other reasons. He spends a lot of time lamenting the industry's obsession with "user friendly" interfaces which obscure the underlying complexity of the problem at hand.
But you could replicate exactly the calculations happening as a LLM computes the next token. Would that be conscious? Where would that consciousness reside?
My view is that LLMs are designed and trained to autocomplete narratives, and that as part of that process they develop an emergent narrative about themselves. Whatever personality traits they exhibit is part of that narrative. Perhaps this is not that different from how humans learn to move in social contexts, but it is not consciousness as it happens in living beings.
This is software trained to produce outputs. It doesn't feel anything.
This is not an argument for true AI being possible, let alone already achieved; it is a warning about depending on arguments from incredulity, which are particularly susceptible to future developments.
Searle's Rooms is a fully rejected and rebutted argument, and Philosophy will never be (and never has been) a serious discipline until it figures out how to actually let go of failed arguments.
idk how legit this is but the summary is my point https://arxiv.org/abs/2206.07682
They can have empathy too. Meaning, you can describe a circumstance with many parties involved, with different motivations and conflicting goals -- and it will tell you how these humans would likely feel about each other and the situation at hand.
To paraphrase the Chinese room argument: Put me in a room with infinite time, paper, pencils, filing cabinets, and give me an English language list of instructions for doing all the neural-net calculations required to run an LLM. Slide your prompt under the door, and, after doing all the calculations, I will slide the next word out, and so I will repeat until the response is done.
In this thought experiment, I'm just following the instructions. So, in this case, what is doing the "thinking" or "reasoning" or "empathy" here?
LLMs can reason, reasoning is logic, and as you said yourself it is derived from what humans belief is reasonable. However, there is more to the structure of language, the way mathematics is also a language that ultimately describes the universe.
I'm pretty sure somebody will create an improved synthetic simulacrum of self awareness in the coming years, suffering and all, just so that we can enjoy better synthetic software architects and product owners with "skin in the game". If that consciousness simulacrum were put in a teddy bear that shares your whole life and gives you decent advice and conversation, for how long do you think you could, in your private mind, deny it consciousness? Worse, what would your human friends think if you were to maim your teddy bear?
That seems relevant to building The Existifier. [0]
That's not really the same as it having actual consciousness, even if people will wind up treating it similarly though
Humans have consciousness whether or not anyone believes we do
Or at least I do. I don't really know about the rest of you. I prefer to believe that you all have consciousness too though
(OK, more charitably, these concepts are often murkily defined, but people react emotionally to any concrete definition because they have in their mind that something being conscious and intelligent implies that it requires moral consideration, and morality is complicated, murky, and generally emotionally driven in a way that most concrete definitions struggle to adequately describe)
And there's a blurry line here. Obviously slavery is abhorrent, and animal abuse is bad, and factory farming often crosses the line into animal abuse and we should accept that chicken will cost ~25% more per pound to have the chickens be able to walk around a little bit... but how do we decide when a computer system is worthy of respect? Convenient not to think about that.
I think it's more that consciousness can be _ignored_ in subjects that you don't care about, rather than that you consciousness can be realistically attributed to subjects that you do care about, but even then people attribute consciousness to "nature" or even the universe as a whole, or if you want to stretch the argument, quite a lot of religious belief is exactly attributing consciousness to abstract entities.
Yes it will gain consciousness. You can't prove it but other people can't prove that it's not conscious either
At best this is redefining "consciousness" to mean something only tangentially related.
To look at it another way: this form of social construction answers the question of "if a tree falls in the woods and nobody is around to hear it, does it make a sound?" with an emphatic "no". It is a sillier version of solipsism.
Suppose I look at another person and think "this other person must be conscious like me". What exactly is happening here?
First, I observe that I seem to be a human. Then, I observe that the other person is a human as well. Crucially, there are similarities between what I am and what they are. And since I am conscious, they must also be conscious, because consciousness must be caused by one of the similarities that we share, or so I assume.
This assumption can be robustly verified for humans and some animals via general anesthesia. Cause and effect (assuming a materialist stance) can be established between the drugs and human(-like) consciousness. But this is a very rough method, and it doesn't work for assessing a machine since machines can't be anesthetized.
Before anesthesia, it could still be verified in less robust ways. One way is to hit people in the head and make them pass out without killing them. This was feasible and probably commonly observed since the stone age.
But we don't know exactly how anesthetics work or exactly what in the brain is responsible for consciousness, so it leaves the door open to hypotheses that can work on machines. Here's one example: the pattern and temporal sequence in which particles such as electrons interact with eachother in a circuit dictated by an intelligent computer program may have physical similarities with how particles interact in a human brain, and certain patterns or temporal sequences of interaction may be responsible for (human-like) consciousness.
Suppose something like that were to be proven true for humans with future brain research, then it would be possible to test for human-like consciousness in any brain or computer, regardless of architecture. I suspect this is what will happen because inquiry into consciousness has always been trending downwards toward lower-level explanations, so it's probably not going to be fully explainable at the biochemical level. We'll have to go down to physics.
What we have built as of 2026 would, in Mass Effect terms, be considered a rudimentary VI.
Since hearing that, discussions of a machine "thinking" or "believing" something didn't really bother me, even though Dijkstra was adamantly opposed to such language—because everyone in the conversation usually understood that the meaning was metaphorical, like the swimming submarine.
Your brain is just a big set of weights/neurons organized into layers that do back and forward propagation, turns out. You are just statistics surrounded by flesh and bones.
I would hope I didn't have to regurgitate that kind of basic intro to logic copy in a forum named after a combinatory logic.
This statement is no where close to a statement that matches your concerns. It implies “we all know the Economist is an inherently unserious rag… which it is obviously not.
Edit: I am not in anyway trying to be obtuse or reductive, I simply want to understand your statement. Apologies if it comes across that way.
This sounds a bit like depression, I think they’re genuinely concerned for your wellbeing
At what precise level of complexity is a computer conscious? That's the blurry line.
For me, because the Copernican Principle seems relatively undefeated.
> We've become essentially a force of nature, rather than a simple occupying species.
So are beavers, trees, plankton... they all have huge effects on the planet.
Regarding analysis of animal brains vs. human brains, the more we do, the more we find that we have in common with many other "lesser" species.
Regarding us vs. machines, I certainly believe we are just very complex and sometimes easily programmable analog machines.
This is just my 2 cents on the matter. We are pretty darn neat, but at this point I believe it would be folly to think that we are on some different incomparable plane.
Orangutans can drive a golf cart, there are videos of that from the pre-AI era. General anesthetics that work on humans also tend to work on them. There is just a smooth gradient of brain evolution.
We don't know exactly what in the brain gets disrupted during general anesthesia (or a baseball bat to the head, for that matter) that causes consciousness to go away, but inquiry has been trending towards lower-level explanations over time, and it's unlikely that it can be fully explained at the biochemical level. We probably need to go down to physics.
Once we're at the physics level, there will be testable hypotheses that work on machines. For example: the pattern and temporal sequence in which particles such as electrons interact with eachother in a circuit dictated by an intelligent computer program may have physical similarities with how particles interact in a human brain, and certain patterns or temporal sequences of interaction may be responsible for human-like consciousness.
Solipsism or simulation theory type of metaphysical positions remain undefeated, but they don't matter for practical purposes because materialism and its accompanying theories of physics have the same predictive power about what I will observe in the future regardless of whether solipsism is true (def true: as real as my own consciousness). If I want to be philosophically rigorous, then I will keep in mind that I can wake up from a simulation at any moment. But until then, physics rules.
It may be that consciousness is not some ideal observer entity thing, but is rather made from and defined by the messy goop of neurons that is mostly the brain, which is changing from moment to moment as cells reorganize, so the definition of "I" is constantly changing in a ship of Theseus kind of way. It may be that nobody has the ideal definition of consciousness (or anything, really), like how nobody has the same definition of an apple even if we're looking at the same apple, because the light bounces off of it slightly differently from my angle of view, so our definitions are only mostly similar. This might be why frontier physics is stuck, because we're current assuming that observations/observers can be defined in a universally identical way. If it turns out that all instances of experience/experiencers are unique and never truly identical, then solving the hard problem of consciousness may end up being equivalent to finding a unified theory of physics, because we'd be trying to describe everything one can observe.
And there's lots of evidence that we've hit a million unlikely goalposts. We might've gone through more than one great filter already. Whether it's going multicellular, or having a protecting planet like Jupiter, or having Theia slam into us and giving us a moon for tides, or any number of other weird things.
And that's before getting into our specific evolutionary past. If it weren't for dinosaurs ruling the planet, we might not, as mammals, have evolved our extreme adaptability. If it weren't for them then getting wiped out, we wouldn't have had a planet-sized sandbox where nearly every niche was suddenly open again, allowing us to explode in complexity as we essentially spent all day solving survival puzzles. Even then, all of this could've happened underwater and it wouldn't have mattered, because we never would've made fire.
and that specialness appeared when? in our mammal ancestors? our Big Ape ancestors? Australopithecus? Denisovan? Neanderthal? Cro-Magnon? Or only when we started agriculture? or only during Roman Empire? or only during Industrial Revolution?
That "specialness" is actually the religious "soul" in disguise - the "specialness" is used when the people hesitate to directly bring up "soul" in a supposedly rational people conversation where bringing up "soul" is borderline intellectual faux pas. And i'm yet to hear the answer when our ancestors got the "soul".
Btw, all those debates about AI sentience will become moot in just a few years, around 100T models (about the number of our neuron connections). We already have 10T models, so the wait is almost over. We'd be able to directly ask the 100T model whether it is conscious, and no human would probably be able to successfully argue against the model, whatever answer it would give.
Thst never works. It's not about money.
It's about biology, likely microplastics acting as hormone inhibitors. Sex drive and reproductive drive are not the same.
We know next to nothing about microplastics - pseudoscience can wait at the door.
Take this statement: "consciousness emerges from sufficient complexity in any system". Neither you or I can prove it right or wrong, but we can both make an assumption that if it was right, then there would be a way for an LLM to be conscious.
I'm not arguing that they are, I'm saying being this confident that they are not is the exact same as being confident that they are.
Do not presume to understand other people's concerns based on one or two sentences. I don't know how to explain that in an online conversation, this is more a learn to interact with real life people kind of thing. I would assume older generations still behave that way, because they are more authoritarian.
I could talk about why the 'implication' is not necessarily problem and about why what is 'obvious' to yourself should not be taken for granted when talking through disagreements with other people. But that would take more time and I think a well-rounded college education is supposed teach people these basic humanities skills, not something that can be sorted out online especially these days.
I have been reading more and more; I guess looking ot knowledge as some eort of ends vs a means to an end for once. Camus, Stokely Carmichael (Kwame Ture), Dostoevsky's underground man etc...
Camus: The universe was always going to stay silent when you demand meaning from it; the answer isn't a better metric, it's to push the rock anyway ("imagine Sisyphus happy").
Dostoevsky [underground man]: Refusing to be a piano key in someone's optimized Crystal Palace
Stokely Carmichael: "Racism is not a question of attitude, it's a question of power. Racism gets its power from capitalism."
My conclusion: AI didn't take our meaning, It made me realize I have always been looking in the wrong place and thus reacting ignoratly when I felt "denied" this thing I feel entitled to.
Love.
Edit: Love; paradoxically, ive felt both entitled to and unworthy of so I'm not sure WTF to do with that.
1. "LLMs have empathy since they write empathetic things."
2. "How do you know its their own empathy?"
3. "LLMs are authors, just like humans are."
4. "How do you know that they are that kind of author?"
5. [GOTO 1]
I don't know what you ate today. It would be insane for me to confidently claim that you ate two bananas and a pizza today.
The burden of proof is on the person making the claim. That's the basis of empirical research.
I've read a number of panpsychist publications, yet I have yet to encounter an author who asserts with certainty that stones are conscious exactly in the same way human beings are.
The more common panpsychist interpretation, that everything exists on a spectrum of consciousness, with stones at the low end and humans at the high end, seems like a fairly reasonable ontological hypothesis to me.
As it is, it would be insane for me to confidently claim you are conscious or even that you're human, as an LLM could have written the same message.
The philosophical zombie does not satisfy occams razor because it asserts an unmeasurable property that has no effect on the outcome.
If it acts like it has consciousness, then assuming it to have consciousness is more scientifically rigorous, than assuming for it not to have consciousness, because the former requires one less variable.
So why keep discussing?
Take the radical skepticism pill and embrace epistemological monism - the idea that the only real truth is that all things are interpretations.
I'm being a bit pedantic here, but no I don't think that "I am conscious therefore you must be too" is true
I think you can claim that it is most reasonable to assume other people are conscious but I don't think it reaches the level of proof
So while I know I am conscious, I do not know that everything I experience aren't just impulses fed to my brain by some external force. And I can't prove it either way. I have no way to test whether or not I'm a brain in a jar because the very thing I would use to evaluate the evidence is the thing that is potentially being manipulated.
However, slapping a solipsist sees them abandoning their philosophy rather quickly.
But I agree with you. We can't prove it, but "other people are as real and conscious as I" is a reasonable assumption to make
Not true. This is "provable" aspects of you and your personality can be changed by physical changes to the brain. There are thousands of instances of this including experiments that verify this.
While you can't "prove" this simply by observing this phenomenon among other people that level of "proof" is impossible. You technically can't prove anything and what we see with physical changes to the brain visibly influencing personality and behavior meets a reasonable criterion without getting too pedantic.
It's trivially easy to come up with definitions of consciousness that support your point. But there isn't one of them that it is trivially singular and above all others from a scientific point of view. If you say, "I know consciousness when I see it," you are just making the point that consciousness is attributed by the beholder. Idem if you affirm "I'm conscious". If you say, "only humans are conscious", you are socially attributing consciousness. Now, I don't want to be argumentative, I honestly haven't been able to figure this matter in a decade of armchair internal debate[^1], and it seems I have some company.
[^1]: Having this internal debate makes me believe I'm conscious, but now that I have seen LLMs in action whose "internal debate" is more lucid than mine, I honestly question that "internal debate" is sufficient.
If you modify the teddy bear enough that's a totally different situation, so let's not confuse the two. In the modification case we don't even need to worry about simulacra, we can stretch it into pajama shape and use an entire human as the stuffing of the teddy bear.
It seems more likely to me that e.g. the individual fibers of the teddy bear are conscious than that there is a consciousness associated with the teddy bear as a whole. And I don’t find it particularly likely that the fibers do.
Like, by what mechanism is the teddy bear “one thing” in a way relevant for consciousness to be associated with that thing.
When people have stuff happen to their brain, this has an effect on their conscious experience in a way that isn’t true of e.g. their hair being cut, or their nails being clipped. In this way, the consciousness of a person seems associated more with their brain than to parts of their hair or nails. But, the way that a persons hair and nails is related to the rest of a person’s body (including their brain) seems more similar to how one bit of fluff is related to the rest of a teddy bear, than the way a part of someone’s brain is related to the rest of their body (including the rest of their brain).
Like, [bit of fluff]:[rest of bear]::[bit of hair or nail]:[rest of body (including brain)] is a better analogy than [bit of fluff]:[rest of bear]::[bit of brain]:[rest of body (including rest of brain)] is, and so, in the same way that removing a bit of someone’s hair or nail in a way that they don’t notice does not seem to alter that person’s conscious experience, while removing or altering a bit of their brain very much can alter their conscious experience, I would expect that removing a bit of a teddy bear would have no impact on any conscious experience it might have, and this goes for, every part of it? Like, any way that you cut it in a (n/(2n+1)), ((n+1)/(2n+1)) split, I would expect that to be more like the “removing some hair” to whatever conscious experience it might have, and so, I expect that it probably has no overall conscious experience?
I find the idea that a single thread has an overall conscious experience to be a little more likely than a teddy bear having one. Like, if you cut a thread, that’s always a pretty fundamental change.. idk. Again, I don’t think that’s like either.
Otherwise don't be stupid. Come on.
That’s not to say that I think the Chinese Room Argument is convincing. I don’t think it is. Or, I guess it might demonstrate something, but not what it aims to show.
I would argue that if, say, the person who has memorized this program and mentally executes it, and if this program also includes actuating some levers to control a mech or something, and the program (running via the person mentally going through the instructions by rote) takes actions to pursue some goals (e.g. purchasing some wood by conversing in Chinese and building a tower with that wood), while the person on whom the program is running wouldn’t know how to order a cheeseburger in Chinese,
Then, the program running in that person’s head might well understand Chinese even though the person doesn’t. (Whether or not the program actually understands Chinese, I think probably depends on the specific of the program in ways that I don’t think anyone knows. I don’t expect that any program that would understand could be small enough that a human person could memorize it and execute it by rote in this way, but this is a thought experiment and such issues can be disregarded I think.)
Critics immediately point out: yes, there clearly is still understanding, it's just in the system itself -- the program. The fact that the person does not have the understanding doesn't mean the understanding is gone.
His response to this is: but what if the system was the person's brain!?
Okay, so either it doesn't address the criticism at all (there is still a system of rules that contains the understanding; who cares whether that system is internal or external to the person's brain?) or it completely destroys the original premise of the argument, which is that the understanding is not in the person! You can't put the understanding back in the human agent and still claim the human doesn't understand what they're doing! His entire argument relies on the fact that the human agent obviously doesn't understand Chinese, and then says, actually, their brain contains everything it needs to know to have completely fluent conversations in Chinese. So the person does understand Chinese!
> How does the person having all the instruction books memorized and whatnot imply that the person understands Chinese?
How does a person's brain containing all the facts, information, and skills that it needs to have a fluent conversation in Chinese imply that the person understands Chinese? How does it not!?
Notice that the man while having memorized the rulebook can't convey any fact of his childhood in Chinese. He can only convey facts encoded in the rulebook, perhaps of a hypothetical childhood. The point is there are different ways information is stored in the brain; one way is through rote memory, the other is something like semantic memory. The rulebook encoded as rote memory doesn't endow his semantic memory contact with the meaning of the symbols. Hence the man doesn't understand Chinese.
This doesn't render the Chinese argument successful. The system's reply still stands when understood properly. I like to call it the virtual mind objection. Physically executing the rules in the rulebook just instantiates a virtual mind. Whether the man is reading the rules from the book or just pulling them from memory is irrelevant.
Because they can’t use Chinese to speak about things they encounter in the world?
Like, someone asks them in Chinese whether they are hungry and want some food, and they answer based on the program, not based on whether they are actually hungry, because they don’t understand the words in a way that lets them use them as a person who understood Chinese would.
Again, I don’t mean to say that in such a situation that there is no understanding of Chinese happening in their head, only that this doesn’t constitute that person understanding Chinese.
It seems to me very strange to want to say that the person with the simulation in their head understands Chinese. They can’t do the things that understanding Chinese lets people do.
Perhaps the program the person has running in their head is even a person, a moral patient and a moral agent, and one who understands Chinese. But the person who is running (by rote) the program, doesn’t thereby understand Chinese. They may “contain understanding of Chinese”. The understanding is definitely happening in their brain. But they don’t understand Chinese.
They don't, they have simply memorised a translation function which takes Chinese as input and returns Chinese as output. They don't have facts or information about the Chinese language - they don't even need to know what language they are speaking!
When the person reads 汉堡包, do they understand what that is?
Assuming you don't speak Mandarin, do you understand that word? If not, how could you claim to understand Mandarin? Even if you could use the translation function to speak it, you still don't understand it. Remember that the translation function is Mandarin -> Mandarin, so if you don't understand Mandarin you cannot use it to gain an understanding of the word.
If it isn't that, then we're talking about some non-practical definition, in which case, does that even matter?
I think Occam's razor is sufficient to reject this as a concept. Panpsychism would introduce a lot of complexity into our understanding of the world without really increasing our ability to explain any of it
I don't feel the need to make a stronger case than that because frankly it strikes me as more like mysticism woo than any sort of real serious philosophy
Functionally,I'd say a key difference is that it's probably easier to perform general computation with a computer than to achieve the same goal using a stone.
Tendentiously redefining the word "simulation" in an attempt to avoid the awkward cases is not going to save an argument that was dead on arrival.
something I once wondered about is what a soul can be and what implications it would have. the conclusions I drew is that the entire concept was incoherent. if a soul is so intrinsically linked to the brain (experience) such that it always goes along for the ride even into the abyss (eg. dementia) then it by itself wouldn't have any value even if it existed.
If in responding to its inputs it experiences consciousness (=lights are on, experiences senses, perceives qualia), then you’ve built a brain. If your model simulates all the processing of a but does not have these then you have what’s fairly called a simulation of a brain. You can assert there is no such thing but good luck getting me to believe you.
No such thing as what? As far as I can tell, I have asserted no such thing as there being no such thing, whatever it is.
No, because the simulation is actually missing an important aspect, namely the causal part that exists with the physical material but not in the simulation. It’s actually that it’s missing these components that makes it a simulation and not the real thing. But for that you need the physical material.
So why would it be different for the brain? If nuclear energy can’t arise from abstract relationships, why would consciousness or anything else?
There are many differences between Enigma-machine simulation and nuclear-reactor simulation, but the one that is relevant here is that the former's inputs and outputs are abstract information, and we have very good reason to think that a brain's biological purpose (in the sense of what the organism needs it to do) is, like the Enigma machine's, abstract information processing in the broad sense.
Because consciousness isn't something the brain produces like a hormone or an excited particle. Consciousness isn't an output of the brain that is somehow separate from it, it is the manner in which the brain processes sensory information and makes decisions to produce output.
The causal context in which brain dynamics occurs is orthogonal to the relevant properties regarding consciousness. Consciousness is a state characterized by certain kinds of information processing, namely that which carries relevant semantics of the organism encoded in a way that maximizes effective control by the executive center. Consciousness is the rich semantic space in which the executive center engages to affect highly competent control of the organism. It is substrate independent information processing.
There's also the question of input output range.
When you hear a certain set of symbols in a certain context you jump.
Because in the thought experiment, the translation function (which is what you mean by the book) is f(Chinese) -> Chinese. If you don't have knowledge of Chinese, it won't help you.
We can also play a little game. I’ve invented a language called Kejdoke.
Me: Jeheixis ownc j’Lm hys bjækf¿
You: Kloften¡’by
You should be able to tell me what this exchange was about. Since any reasonable definition of understanding for a language would be to do this, and you have the same knowledge of Kejdoke as the person in the thought experiment has about Chinese. If you can't tell me what this exchange was about, the person in the thought experiment could not understand either.
Are you conscious on a nanosecond timescale?
I said nothing of the kind. I can only assume your question in response to my comments mean you think a powered computer is alive. I'll challenge that on the basis that a computer can generally maintain its state without power just fine. Shut it down, unplug it. Wait a day, a year, a decade and plug it back in. Assuming the hardware didn't degrade to the point of failure you can turn it on again and it will be in the same state you left it in before you unplugged it. Is it alive while its unplugged? Or is it simply not alive whether its plugged in or not?
What is this then?
Update: I see ratelimitsteve agrees with that assertion.
You say that the physical hardware your enigma simulation runs on provides the causal element. But the simulation of the nuclear reactor also runs on physical hardware. Yet it obviously doesn't provide the causal element, and neither does the enigma machine simulation's hardware. There are no spinning gears for example. Hence why you and I both call it a simulation, I think we're in agreement here.
I think you could argue that while the actual simulation of the enigma machine is not an actual enigma machine, the essence of the enigma machine is not grounded in physical reality, it's the abstract relationships we're talking about. And I could also agree. The essence of the enigma machine are it's abstract inputs and outputs.
But now about the brain. You said
> When you have digitally modeled a brain such that the model responds to its inputs as a brain does, then you have built a brain
The issue is that you have not built a brain, you have built a simulation of the abstract relationships of how a brain works. The brain is obviously more than that, considering it exists in physical reality. And thus the claim is that the essence of the brain is the abstract relationships, correct?
The problem I see is that consciousness is a physical phenomenon. In fact it seems like it's the magnum opus of phenomenon, in that all our experience comes from our consciousness. And you have to then bridge the gap between the simulation and physical reality. At most you can say you "simulated" consciousness, but questions about qualia remain.
Anyways I'm not trying to attack you or your views, I'd love to hear where I go wrong in your mind on this.
What I actually wrote was this: "Then, presumably, you will also hold that when you have digitally modeled a brain such that the model responds to its inputs as a brain does, then you have built a brain." This has to be read in the context of the post by ratelimitsteve to which I was replying: I am not saying that I think that when you have digitally modeled a brain such that the model responds to its inputs as a brain does, then you have built a brain, I am saying it is something that ratelimitsteve would have difficulty denying while being consistent with his distinction between simulating and building. Personally, I would not say that simulating a brain to this extent amounts to building a brain, as that seems to me to be unnecessarily awkward usage that I feel will lead to confusion without making anything clearer. I would still call it a simulation, albeit one that might justifiably be called functionally isomorphic to a biological brain if it were good enough.
I would describe the relationship between an Enigma machine and a digital simulation of one in the same terms, and if I possessed an actual Enigma machine, I would not willingly swap it for a digital simulation!
With that said, let's go back to your second paragraph:
"You say that the physical hardware your enigma simulation runs on provides the causal element..."
Yes. The design of the physical Enigma device contains an abstract specification of how outputs are to be generated from inputs, such that when the machine is used, physics ensures that the correct output for any valid input is generated as a consequence of the causal relationships between its components. In the same way, the software of its digital simulator contains an abstract specification of how outputs are to be generated from inputs, such that when the program is run on a sufficiently capable computer, physics ensures that the correct output for any valid input is generated as a consequence of the causal relationships between its components, once the program has been loaded.
"...but the simulation of the nuclear reactor also runs on physical hardware. Yet it obviously doesn't provide the causal element..."
Correct, insofar as a computer is not capable of undergoing nuclear fission, regardless of how it is programmed.
"...and neither does the enigma machine simulation's hardware. There are no spinning gears for example."
No - these two cases are not equivalent. Nuclear fission is required for a nuclear reactor to function, but rotating gears are not required for encrypting or decrypting a message according to the enigma machine's abstract specification; they are just how the specification happens to be implemented in the machine.
"...Hence why you and I both call it a simulation, I think we're in agreement here."
Well, we both seem to be in agreement as to what to call it, but apparently not over why we think that.
In next paragraph ("I think you could argue that..") you say you could agree that the essence of the enigma machine is not grounded in physical reality, it's the abstract relationships we're talking about.
I am OK with that so long as it is understood that some sort of implementation is required for anything to happen.
Now we get to the misunderstanding of my reply to ratelimitsteve, covered above, and after that you go on to say "the claim is that the essence of the brain is the abstract relationships, correct?" My response to this is the same as with regard to the essence of an Enigma machine: I am OK with that so long as it is understood that some sort of implementation is required for anything to happen.
The next paragraph begins "The problem I see is that consciousness is a physical phenomenon...", and I tend to agree with the claim (though not that it is a problem) again in the sense that some sort of implementation seems to be required for anything to happen. I don't claim to be able to prove this, let alone explain how it works in brains, it just seems to me that there's no plausible alternative.
Then you write "In fact it seems like it's the magnum opus of phenomenon, in that all our experience comes from our consciousness..." To me, this appears to be a version of Cartesian doubt, about which I am extremely skeptical, in this sense: I think it is tendentious for anyone to invoke it only when it greases a sticky argument, while otherwise acting, speaking and arguing as if our sensory experiences give us reliable information about an external world.
Finally, we get to the crux of the matter: "you have to then bridge the gap between the simulation and physical reality." Here you just declare there to be a gap between the simulation and physical reality, but to my mind, nothing you have written up to this point provides any justification for thinking there is a gap that needs to be bridged. As best I can tell, you feel that by emphasizing the abstract nature of what you call the essence of a simulation you have divorced it from the physical, and my response to that has been, consistently, to point out that these abstractions do not get anything done unless they have some implementation, and there is no evidence of those implementations being in any sense non-physical.
But I will point out we don't have a way to prove anything is conscious, not even a human.