- Anthropic told everyone Mythos was dangerous because it's proficiency with biologics and cyber security

- Anthropic didn't release Mythos like everything else. They released a neutered fable. They didn't get rid of Mythos

- Anthropic opens a lab in SF

There was always a quesiton of "will the labs stop releasing their models and start building around them instead?" Yes - they already have. Anthropic is a biologic and cyber security company, in addition to intelligience.

Personally I wonder if they've been holding back a lot. Opus 5.5 was a good release after a little stagnation. Open AI releases good models and everyone says Anthropic sucks and -- Oh would you look at that - a better model finally and all of a sudden.

growing up I always assumed that everyone would grasp some aspect of game theory intuitively, I still remember the day I found out game theory was a thing - it was like finding out that someone successfully systematized common sense.

most people take the things people say as if they were worth considering. signals without cost are only useful as knowledge of what the signaler wants fools to believe.

> If recent news is to be believed, Anthropic culture even today seems to lean heavily on this effect

> In my opinion it has been the leading factor in getting and retaining the best employees who are often very worried about humanity dying to super intelligence.

I had not considered this viewpoint but this makes a lot of sense. A lot of Anthropic employees truly believe this and callout emphasis on safety as a key reason they work there. Now, Dario's post "Pacing the frontier" makes even sense -- it is as much for his employees than it is for the rest of the world.

It also shows the same reasoning error mode many criticisms of a precautionary initiative or intervention to a problem:

Assuming that a problem whose trajectory was at a certain place when the initiative began has failed simply because it isn't solved on their own wished for timeline or standard of success, or that it wasn't meaningfully changed from what itnwod otherwise have been.

What happened to realizing there are hard problems, that different things may need to be tried, or that those things tried were partial but not complete solutions?

But I think they both are thinking the same thing which is... "We are gonna run out of money at this pace."

However! If one of them blinks and turns off the money faucet before the other, they might fall behind. Falling behind is to forever lose. And if there's one thing that a CEO hates, it's losing to a rival CEO.

So what they want is to get someone, anyone, to put the brakes on their rivals and them at the same time so they can both Not Lose, and Stay Alive. Under those new rules, they are confident they can win. And by win, I mean beat the other AI CEO.

That's it. It's always about personal incentives. Get out of here with that safety BS. These guys just want to win.

* AI Labs hit the scaling wall. They need either new techniques, or vastly more powerful hardware to advance further.

This explains, the miraculous incompetence of AI labs in securing sandboxes and figuring out "alignment."

So they are between a rock and a hard place. They need limitless VC money because they cannot operate otherwise, and they do not have the capabilities to go further. The scare tactics and the "pacing the frontier" makes perfect sense then; they can IPO on the assumption that their ridiculous balance sheet doesn't matter because they are holding back. Because they are in control. The regulatory capture would be double whammy if they can manage it.

Open AI already said they have smarter models, and Opus 5.5 is rumored to be "taught" by a "teacher" model already; they are essentially distillations from bigger models, that both labs probably cannot economically serve to the public, due to hardware simply not being there. And, most of the improvements are not at the model level, but at the agentic glue level. Labs are getting better at RL'ing the models for agentic use cases, but the inherent flaws are still there. Models still have trouble with locality in writing for example (bunch of research on this that shows model size is the determinator), and agents are the bandaid over that.

And in the meantime if one of the labs makes a breakthrough, they'll push with all they have, because why wouldn't they? The idea that current LLMs can actually go rogue is just hilarious; in all cases, agents are being led by (deliberate) incompetence.

Pacing the frontier and the scare tactics will be seen as new generation's snakeoil tactics, perhaps will be called a flavor of AI CEOing or something.

An industry trying to figure out how to cut expenses, reduces the pace of training under the guise of "safety".

Coincidence?

It's also worth remembering that there is zero percent chance that entities like the US military are going to be pacing anything. What Amodei and his ilk are aiming for is a highly regulated industry where they control the political barriers and the ability to sell SOTA model access to state actors that have a monopoly on violence. It's the worst possible situation for consumers and citizens. Thankfully I don't think they can put the cat back in the bag and Chinese and other models will keep progressing as a counterbalance to the techno-fascism Anthropic is aiming for.

Even if the leading US labs could agree amongst themselves to a coordinated slowdown (and this would likely run afoul of antitrust law), you still have Chinese labs who will catch up to the frontier eventually. To solve this, you'd need some sort of international agreement.

That's why Dario and others are saying loudly that we should pace the frontier in hopes that our political leaders will take up the issue and do something about it. We'll see if that happens...

The problem with this analogy, and really this whole post in general, is that it just doesn't recognize that these are businesses which are (at least from some perspectives) producing real value. This article makes it sound like the business model of OpenAI and Anthropic (etc.) is based entirely around building a doomsday device. In reality, the "promise" of AI is that it can greatly improve many people's lives. It's just that such power, wielded incorrectly, can be dangerous.

If you want to fix this analogy, you'd have to choose an activity which isn't effectively only detrimental. For example:

>Imagine your best friend is a professional bodybuilder. They keep telling you how the sport will one day kill him, and he promises you he's doing the best he can to minimize those risks. He even spend a bunch of his time on TV and writing blog posts about the harms of professional bodybuilding, but even more times talking about the benefits of bodybuilding. But every time you meet him, you just see him bodybuild, and he seems to be increasing how much he's in the gym, eating his special diet, etc., every day.

Would you be confused by how your friend is behaving? It is well known that bodybuilding can be dangerous. It has been the reason for late-stage crippling of bodies as well as very early deaths. But your friend isn't going to stop because there are some potential risks if you don't handle the activity safely. They like it. They make money from it. They're doing something they think is productive. They can recognize the risks, and even do their best to highlight those risks and try to mitigate them for themselves and others, all while fully embracing the activity.

Regardless, people like the author don't seem to acknowledge that there are good things that come with this technology. They don't seem to acknowledge that a single company can't purposefully stop development if other companies aren't obliged to do the same. They don't seem to acknowledge that "pacing" isn't the same as completely halting all development immediately.

And I'm not saying I'd trust what any CEO says at face-value, let alone the CEOs of these AI companies. But it's also obtuse to assume that everything they're saying is a clever scheme to trick the public into acting against their own well being. As if that's a tenable strategy in this context.

If these companies are all asking to be regulated, then they should probably be regulated. They probably shouldn't be able to dictate how that works, but it's really not unbelievable that they see a serious risk in their own well-being if this stuff goes unchecked: all it takes is for one truly catastrophic AI event to occur before they all get extreme regulations even if they, themselves, are acting safely. It's in their best interest to slow things down, but only if everyone slows down at the same time.

I largely think that all posturing from the labs about slowing down and deeply caring about safety is done in order to retain and calm the employees who they are dependent on to keep pushing capabilities to get to AGI. If it was not for a big contingent of employees pressing them (increasingly publicly), they would make zero public acknowledgments of risks at all.

The last half of the article is great and worth reading. Really wish the first half of the article didn't immediately apply the Godwin's Law footgun.

EA cult members do not take this very real, measurable, non-speculative danger seriously. Consequently, I don’t think they should be trusted or consulted on any subject of any importance.

Game Theory as a field is actually very complex and often reveals dominant strategies that would never occur to someone as the "common sense" approach to a problem. It was arguably created to solve for problems where there was no obvious correct answer, even for a very intelligent person.

No matter how many people think press release X is obviously wrong - 80%, 95, 99% - it'll always be "the comments section criticizing the article," or "bloggers sharing incisive criticism."

It seems even otherwise intelligent people will cultivate as much ignorance as possible to try and bend reality.

I read the whole thing as coordinated behavior to reduce the breakneck pace of 2026.

Anthropic's revenue is up 50% in the past two months. They're not hitting a wall.

That is not the promise of AI. The promise of AI is that it can automate knowledge work.

Nothing about these models or the productivity they can bring is related to benefiting others. That would only come about from political control that forces the benefits to a large swath of people.

As it stands AI looks like it’s going to decimate the middle class even further as white collar work gets obliterated and the owners of the AI companies hoover everything up.

That's not exactly true.

"Climate change matters so much, to so many, not just because of the suffering and injustice it’s already causing, but also because it’s one of the few issues that has obvious potential to affect our world over many future generations. We think safeguarding future generations is a key moral priority, and should be a crucial consideration in prioritising problems on which to work.

...

...climate change will be hugely destructive. We’ll see floods, famines, fires, and droughts — and the world’s poorest people will be affected the most.

...

...climate change’s impacts will still be significant – it could destabilise society, destroy ecosystems, put millions into poverty, and worsen other existential threats such as engineered pandemics, risks from AI, or nuclear war. If you want to make climate change the focus of your career, we include some thoughts below on the most effective ways to help tackle it.

...people are right to be angry that too little is being done.

...

Working on this issue seems to be among the best ways of improving the long-term future we know of..."

https://80000hours.org/problem-profiles/climate-change/

A big part of the reason EA doesn't focus more on climate change as a "highest priority area" is simply that many people are already focused on it, and it is therefore not an especially "neglected" area.

And then we raced ahead straight into materializing the x-risk everyone thought is still a few decades away, speedrunning through all the mistakes LW folks itemized and worried about over the past two decades.