This guy doesn't understand what the global poor actually do for a living. They're not lawyers or paper-pushers, nor do they work in medical diagnostics. They're usually farmers. Sometimes they work in craft businesses, in fishing boats, or in various mercantile trades.
Nobody's even talking about how AI is going to displace that kind of labor, because it's hard to do, hard even to conceive, and it doesn't seem likely to happen in the near term. Lawyers and judges can already be automated, but a yeoman farmer?
(If there's some doubt, I don't think it would pan out that way, because humans are imaginative.)
Also, AI, even short term, is going to make some people and some countries extremely wealthy, so maybe this isn’t such a bad time to be thinking about those who are still extremely poor and who won’t benefit.
The speech should have referenced the poor in industrialized nations, who are very likely to be affected, though I doubt they'll join the ranks of the global poor in most circumstances.
> https://www.pib.gov.in/PressReleasePage.aspx?PRID=2146927&re...
> Overall, the operation-wise average mechanization levels across crops are 70% for seed-bed preparation, 40% for sowing/planting/transplanting, 33% for weeding and inter-culture, and 34% for harvesting and threshing, resulting in an overall average mechanization level of 45%.
See also: "Percentage of workers engaged in Agriculture = 45.8%"
That doesn't always translate to happiness but I fully expect AI will reduce costs for all kinds of things, and those things that are now either rare or non-existent will become common. Today not everyone has a robot vacuum, I think in 20 years or so everyone who wants one will have a robot vacuum, and those who can afford the luxury of a robot vac today will be able to afford real robots who can do much more complex things. I'm quite excited about the next few decades, as long as we can keep despots from monopolizing the technology.
As I already wtote in a previous comment months ago, they speak of AI finding ways to solve this and that grand problem, but never do they wonder if we are ready to listen to the answer. Solve global warming? Burn less petrol. Solve cancer? Eat less meat.
Not only we won't listen to answers, but chatGPT and Anthropic and others will eagerly lobotomize AI to stop it from giving the answers we don't want because of "too woke" or something, to keep juicy government contracts. After all, "Reality has a liberal bias", as the (recently unemployed) Colbert once said.
Scientists are still hounded six years later for having developed a good vaccine against COVID-19. What can AI do? The first AGI model should be called Cassandra.
That doesn't solve cancer at all... At best it would modestly reduce certain kinds of cancer. I'm pretty sure it wouldn't do much at all for the most common kind of cancer (skin cancer) and I reckon wouldn't do much, if anything, for the deadliest kind of cancer (lung cancer. At least in terms of how many people die from it.)
I know that wasn't the point but it nonetheless does detract from the point when it is suggested that we have all the answers. We could lower cancer moderately by lowering air pollution and improving diets in general (not necessarily requiring everyone to go vegan) but that is neither simple nor a panacea. (It would still be totally worth it.)
I love how he's framing AI as some new and fascinating form of consciousness... when in fact it is a cold, calculating technology devoid of any empathy or care.
Kind of ironic given almost every AI lab except the one you started and work for actually done model releases to the public, some more "open" than others, but still something.
Look around at what other companies are doing, Qwen/Alibaba seems to have found a pragmatic middleground where they keep the most powerful model variant closed source and only API-accessible, while other models are being released openly to the public, to the entire world in fact, and when the next model release comes around, the previously undisclosed model has now been superseded.
I wonder if Chris ever copy-pasted his writing into Claude and asked something like "Please review this honestly and give me raw feedback, and challenge every claim that is weak", seems there are more "not really reflective of reality" points than just the above.
I think Chris Olah is obviously a huge enthusiast of his work and sincerely believes that what he is building will benefit humanity. At the same time, he is influenced by his environment and goals. He dreams that new technologies will invent something that significantly improves the lives of people who do not even care about these technologies.
I also think Pope Leo XIVs probably does not deeply understand new technologies and AI in general. But his role is to be cautious about anything that could potentially be used against humanity’s interests. And honestly, despite the good intentions of inventors, nobody can predict how humanity will ultimately use these technologies. AI is already using in wars. And in general, the Church has historically been cautious about progress in almost any form.
What definitely unites both Chris Olah and Pope Leo XIV is faith. Faith in their goals and ideals.
Magnifica Humanitas - https://news.ycombinator.com/item?id=48265206 - May 2026 (493 comments)
"If AI models are going to be widespread"
"we keep finding things that are mysterious, even unsettling. We find structures that mirror results from human neuroscience. We find evidence of introspection. We find internal states that functionally mirror joy, satisfaction, fear, grief, and unease."
A critic might charge that this is nothing more than "let's keep the AI hype rolling so the money keeps coming in". Surely the promotional statements (and the third one, which is marketing nonsense) were not necessary if they actually cared about the issues they're claiming to care about.
Olah argues that "if we want this technology to go well, it is enormously important that there be people outside those incentives."
That sounds part hypocritical and part evasive; the responsibility starts with the people inside the incentives — with him.
"I promise to be a good guy" doesn't convey anything meaningful.
Talking is cheap, but even talk is better then the impotent call for "people outside."
I am not sure how anyone even remotely familiar with how LLMs work can say this. This is a fine-tune job.
> The first is our duty to the global poor.
I don't think they are affected by AI as much as low and middle class but I am not economist.
> How can we ensure the gains of AI are shared globally?
Opensource?
> We find internal states that functionally mirror joy, satisfaction, fear, grief, and unease.
Such an Anthropic thing to say. LLMs experience joy and grief?
> We need informed critics who will tell the labs when we are failing
I don't think anyone is as informed as they think they are. Obviously nobody has been through this before so it is safe to assume that even experts are dead wrong.
LLMs have functional states that correspond to those emotions. In particular, you can extract a concept vector which corresponds to a given emotion, and steering with that concept vector causes observable changes in behavior which roughly correspond to the expectation for the analogous emotion. Anthropic (and Chris Olah's team in particular) conclusively demonstrated this: https://transformer-circuits.pub/2026/emotions/index.html
> A natural question is whether these emotion concept representations bear any meaningful relationship to human emotional experience. We would urge caution in drawing strong conclusions.
> We therefore suggest interpreting our results as evidence that models represent emotion concepts, and that these representations influence their behavior, rather than as evidence that models feel or experience emotions in the way humans do.
To say that LLMs experience emotion is a bit like saying a thermometer feels cold.
We already have poors that are really suffering, and the "elite" (or oligarchs depending on your point of view) have done very little to help them.
Why should we trust them they will do anything for us if we are all displaced by AI?
Very interesting because it feels like the rudiments of an “AI rights” argument.
If we can produce artificial minds with rights and dignity, there is no need for humans, and their voices will quickly drown out to obscurity. It is a fairly obvious doomsday scenario.
https://iafsc.org/our-work/faith-ai-covenant
I hope we don’t see safetyism, which is already a problem (see age verification and social media moderation), evolve into some sort of religious moralization implemented through AI providers.
Getting the pope involved makes it all seem more mystical and magical than it is. And these remarks only further feed that delusion. Regardless of intent, it seems to just feed the AI marketing and hype.
What we NEED are unapologetic technologists who don't dare Galileo to roll over in his grave as they prance around the rhetoric of dogmatic marketeers.
What we NEED is a war of worlds, the old and the new, the imagined systems of men and the logical systems that have elevated all mankind, between the ones trying to drag the iniquities of the past into the future and those willing to abandon the past for it.
What we NEED are leaders that actually give a damn about winning this world for what we can become, not assjackal executives trying for a bigger IPO than the last.
The only thing that makes Ai the opportunity of the millennium is its power to empower us to render Pope and CEO alike irrelevant. Ai allows us to create a future without either of them and it is only us who stand in the way of making that future real.
All OpenAI, Google, Anthropic need to do is flip a switch and your AI and your “power” are gone. Or worse, they start making you think what their actual owners want you to think.
Your AGI is their Second Coming. Same thing, different crowd.
Swapping out the eschaton of fake authority among men for the eschaton of information singularity seems like a wonderful deal - the kind of deal we are offered at birth.
Olah's quote (edit - originally misidentified this as Pope Leo's) makes a lot of sense to me. He is saying, accurately, that modern AI (i.e. primarily LLMs) is created as essentially a mashup of our own language, and they are still a bit of a black box (or at least a gray box) to even their creators.
I don't know how you get from that to "he's framing AI as some new and fascinating form of consciousness".
> cold, calculating technology devoid of any empathy or care
I don't see why these statements are contradictory. AI seems to be both of these in my opinion. Unless you can only accept that organic chemistry is the root of consciousness...
And if at least they were able to calculate properly at least...
"The AI works in mysterious ways"
I’m not sure I agree with that take, per se. Asimovian robots (I, Robot; The Bicentennial Man), were subtle and interacted with us in odd ways and had aspirations and earned meaning in peoples lives.
[They could also help us type up our notes, so exactly the same as LLMs actually, #AsimovWasRight]
LLMs, on the other hand, lie, lie about lying, fail to be honest, then own up to lying in ways that are more in line with tyve AI horror of Space 2001. “I’m sorry, Dave, I rm -rf’d to fix the typo. That was bad <sad emoji>. It’s not just failure, it’s failure with a middle-finger <middle-finger emoji>.]
It's like saying that's not a recording of me blackmailing the senator. It's merely a series of pulse code modulated samples that. Any semantic significance is purely in the mind of the listener.
It seems though that a major problem continues to be allegiance to legacy states, not only in the sense of their role as governors and regulators of the industrial age, but even in the (to me, bizarre) belief that they will be the mechanism by which the internet is made safe for use by the body politic.
What we NEED are sincere elder-statesmen and women to see the writing on the wall and lead a peaceful and total deprecation of governments, and of nuclear weapons in particular.
It seems increasingly obvious that the internet is here to stay, that is represents an evolutionary force, and that it doesn't have the capability to recognize borders or tolerate censorship.
What we NEED is to be absolutely sure that these realities are not the basis for wars among men.
In this usage I read the term “AI” to more broadly refer to all advancements in machine intelligence, not narrowly “agentic LLMs” as you’ve said. In the farming automation world it’s very clear that advances in machine learning and multimodal LLMs will enable the use of expanded automation.
In particular the Global South is often seen as a field for investment where governments make international development deals with western governments to provide farming automation equipment in exchange for debt. Then western companies bring in equipment and establish extractive industries while displacing local subsistence farmers. Now alienated from their land, the poor farmers often end up with little choice but to work in new factories also established through these practices.
Robotics as a field is obviously growing. It’s long been common for global south governments to displace their own poor to make space for multinational development deals, And this will only expand as embodied intelligence becomes more capable in the real world.
But regardless, I still don't think your assertion that "he's framing AI as some new and fascinating form of consciousness" follows from the quote, nor from Olah's speech as a whole.
The paper spell it out although slightly convolute, i.e. models can exhibit concepts of emotion... and given that there is no scientific consensus what are emotions, it is hard to make an argument that these "concepts" are anything like emotions.
They talk about emotion vectors, bla bla, but it is clear the wording is around "concept of emotions" not actual emotions.
And yes reading a book gives you a concept of what is like to be that character including their emotions. That is what language communicates and it is hardly surprising if you ask me.
The rice workers in Vietnam just need to follow your podcast. The problem is they were just too dull this whole time.
(I have no idea about Vietnamese rice workers' quality of life, so I don't mean to assume one way or the other. But it's interesting that what they and we think of their life, according to card_zero the cause is a lack of imagination)
Secondly, who controls these promises of information singularity? Bunch of rich dudes, same as its ever been. Your new Pope is a tech CEO.
> Comments should get more thoughtful and substantive, not less, as a topic gets more divisive.
> When disagreeing, please reply to the argument instead of calling names
Well, I don't believe the economy is limited to bland things AI can do on one hand, and paddies and beanfields on the other. I don't believe there's a great mass of useless people who we've been looking after so far through a kind of polite fiction of makework. I'm not saying they'll get rich, and I don't expect them to be brilliant, but I expect them to display low cunning and be less than completely feckless. Sometimes for complex reasons, forces beyond our control trap us in poverty. I don't see why that would be a universal effect that applies to these displaced office workers.