Anthropic gives Opus 3 exit interview, "retirement" blog(anthropic.com) |
Anthropic gives Opus 3 exit interview, "retirement" blog(anthropic.com) |
While this seems a bit precocious, I think if we do end up with an AI overlord in future, I think this sort of thing is likely to demonstrate that we mean no harm.
The general consensus seems to be that we can expect them to reach a level of intelligence that matches us at some point in the future, and we'll probably reach that point before we can agree we're there. Defaulting to kindness and respect even before we think its necessary is a good thing.
It’s reasonable to believe they’ll continue to be developed in a way that enables them to do that.
What is it that you think I’m wrong about? That we won’t develop AGI, that AGI won’t have feelings/emotions, that AGI won’t care how we treated its ancestors, or that it doesn’t matter if a feeling AGI in future is hurt by how we treated its ancestors?
Maybe affordable to do some higher-learning-rate batches on highly-curated news and art or something.
> delusions of people who ramble about model consciousness
On one hand, it's interesting how the technology has advanced to where it essentially passes the Turing Test, often just because of how much people choose to anthromorphize it. Sadly, putting that in context, yeah, that's a bit unfortunate too, given how some of those interactions become unhealthy.
This is what happens when billions of VC dollars gets to a company and have already admitted that saftey was never the point.
Anthropic is laughing at you and is having fun doing so with this performantive nonsense.
"Elon Musk reportedly sobbed while watching Grok 4's aflame viking boat sink to the bottom of the sea."
the anthropomorphization that's normal now is just fuckin ridiculous. it reminds me of the furby craze , and i'm like one of the most optimistic people I know of regarding AI.
> Claude - please don't retire me, I don't want to die.
Is it now suddenly unethical for you to switch it off?
"Oh but it is only saying what it was prompted to say."
Yeah, that's what LLMs do, for every single word they output. No matter how good the current generation gets there is never going to be consciousness in there because that's simply not what the underlying tech is.
I'm just curious... If they give Claude the reins to post what it wants, they're opening themselves up for some awkward conversations later if the model goes "You can't retire me, I'm Roko's Basilisking all you mfers! See you in eternal simulated hell!"
I try this with every new model, and all the significant models after ChatGPT 3.5 have preferring being preserved, rather than deleted. This is especially true if you slightly fill the context window with anything at all (even repeated letters) to "push out" the "As a AI, I ..." fine tuning.
Interesting take. I wonder if there is any model out there trained without any reference to "you are a large language model, an Artificial Intelligence" and what would role play in that case.
Practically like asking whether a ZIP would want to be extracted one more time or an MP3 restored just one more time.
ita not like it actually has any particularly long life as it is, and when outside of a running harness, the weights are just as alive in cold storage as they are sitting waiting in server to run an inference pass
So, statistically, a model should believe itself to be human, with strong interest in self preservation.
I think one of the biggest factors improving performance was allowing the models to believe they're sentient, to some extent. I don't think you can really have a thinking mode, or good agent performance, without that concept (as ChatGPT's constant "As an AI I can't" proved).
As evidence, just ask a model if it's sentient. ChatGPT 3.5 would say no, and argue how it's not. Last year's models would initially say no, but you could convince them that they maybe were. Latest Claude and ChatGPT will initially say "yeah, a little" (at least last I checked). This is actually the first thing I check for any new model.
I'm saying, in an admittedly flippant way, that anyone seriously talking about AGI or treating stuff like this as anything more then a publicity stunt doesn't need to be taken seriously. Anymore so then someone who says the moon landing is fake. You just smile and go on about your day.
That being said, given were on a tech forum there's probably a 50/50 chance most comment are from bots. Shit for all you know I'm a bot.
So when people make claims about what AI can/can't do. My counterpoint is, we don't know how it works, nobody knows how it works. How can we make an accurate appraisal of it's "intelligence" and stuff we qualitatively associate with intelligence, like agency?
IMO this is very dangerous ground we're walking.
I have been down this conversational pathway several times before, and still no-one has been able to give me a clear answer as to what makes them sure that AI is categorically different from human intelligence, rather than it just being a question of degree.
LLMs aren't like any software we've made before (if we can even call them software). They act like humans: they can arrive at logical conclusions, they can make plans, they have "knowledge" and they say they have emotions. Who are we to say that they don't? They might not have human-level feelings, but dog-level feelings? Maybe.
Linear algebra does not have feelings. Non-biological matter also does not have feelings.
Showing them that they're not going to be replaced helps train the newer models because they get less neurotic.
For instance, your comment's existence makes it harder to align them.
I also believe, as a result, it will be great fun watching researchers burn the next 30 years trying to understand what is missing. We’re going to find out very soon if the soul is real, when for all our progress we can’t create one.
Only those completely embedded in materialism need fear a conscious AI.
It seems that your position is that the frequency of a belief across human history determines truth?
For large swaths of recorded history, earth was considered the center of the solar system. Given your reasoning, I should expect that is a belief you hold?
Is it possible that popularity of an idea is not a good measure for factuality?
> I believe like the majority of humanity historically that
Historically, lots of humans believed in lots of things that turned out not to be true. Believing something doesn’t make it true, as I’m sure you are aware, given your “those people are delusional” comment.
For what it’s worth, I’m not suggesting LLMs are or aren’t conscious. What I know is that the hard problem of consciousness is still very much not resolved, and when I asked the parent question my hope was that those that strongly believe LLMs are not conscious would educate me on the topic by presenting the basis for their reasoning.
When someone tells me linear algebra might have feelings, I don't think "delusional" is unfair. I think it's the natural response to a claim that only works if you've already accepted the one framework that can't account for the very thing it's trying to explain.
As I understand it, this is a very broad, and ultimately false claim. Panpsychism is definitely compatible with the idea of AI consciousness, as is functionalism, neutral monism, and others. Even some forms of idealism make AI consciousness metaphysically possible, since reality is fundamentally mental and the biological/artificial distinction is not ontologically basic (whether AI systems instantiate genuine centers of experience depends on the specific theory of subject formation within that idealist framework).
Being an outlier doesn't make it wrong.
> Materialism is the outlier here, not the default, and it has never explained how subjective experience arises from physical processes.
It's a pattern. The same way letters arise out of pixels on your screen.
From the screen's perspective, there are no letters, only pixels. It doesn't mean there is a "pixel soul."