Anthropic's Safety Superpower(stratechery.com) |
Anthropic's Safety Superpower(stratechery.com) |
The bottleneck is compute and data, not the model. That's why they could only gate it for a bit. The ITAR thing proves it: no nationality controls in place, so the only option was killing the whole thing. Not exactly what an all-powerful gatekeeper does.
Try running the latest OS models on a normal Mac or PC. Claude Fable and Mythos are systems not just pure models.
And of course marketing. Don't believe the hype.
I think Claude is often times underwhelming. Security concerns are also a concern companies have a blond spot for. The really toughest pro security (Yes, pro! Totally different framing!) company I know is Google after all.
What I can companies advise to do is, really having more than just bug bounties but a professional hacker team that does nothing else but attacking them the whole day and night 24/7. This needs to be coordinated with the government otherwise you might sound an alarm and will be SWATed for doing good. And I would pay them huge sums since the risk and fallout warrant such a treatment, not the standard wage.
Hackers are the real deal, not AI. Proof: Hackers using AI.
It can be done through the magic of SSD offload. The worst case involves seconds-per-token speeds, but that's OK if you only care about low volumes of slow unattended inference, which maximizes utilization for the hardware.
(The real worst case, where you're streaming the whole model from the cheapest storage you could feasibly think of, involves multiple minutes per token for a single inference, or even hours per token batch if you're doing many inferences in bulk. That's a lot less helpful, so there's a space for smaller models at the edge, even for unattended workloads.)
AFAICT … despite saying you “disagree”, you appear to be agreeing with the parent comment that the model is less important and compute (all that complex infra) and data (also complex infra) are more important.
> I disagree. It is not the model alone. It needs a system which capitalizes on it. And this is very complex. Hardware, software, architecture - it takes a lot to get it right.
What do you disagree with exactly?
But is that last part actually true though? Sure, there might be 600B+ models available for download and local inference if you have the hardware, but does the users who use Anthropic switch over to those even if they're available even as hosted models? Seems like some do, most don't, Anthropic and Claude remains very popular among the people who use LLMs, there is no denying that.
I'm currently spending $200 for Claude. That's around my maximum that I can afford. I could stretch that to $500 I guess. But I saw reports of people spending tens of thousands of dollars with Claude API. That's certainly outside of my budget.
So if/when Anthropic decides to stop subsidizing subscription (if they ever do that thing, I still not sure about that), I'll certainly look at the other options. And available "open weights" LLMs hosted by someone will be my first pick. Right now Claude 4.8 feels very advanced, but things move very fast...
Only because someone else is paying the bills. I use Claude Opus at work because my employer pays for the tokens and encourages me to do it.
At home, I use DeepSeek Flash. It's not as good, but it's maybe 0.7 quality for 0.001 cost.
Not for now, but how long before we have KYC regulations concerning LLMs?
The amount of tokens required to properly distill a frontier model is so large that by the time you could consume the # of tokens you would either be banned for extremely obvious abuse or a new model would be released, rendering your efforts less and less valuable over time. Intelligence is not a linear thing. Being behind just a little bit can have exponential consequences.
Isn't "distillation" of another provider's model exactly how these models got training date in the first place: Massive amounts of the written word + Prompt -> Answer. Why wouldn't distillation produce similar "reasoning" in the new model? It's just inputs and outputs.
This has been covered before: https://aisle.com/blog/ai-cybersecurity-after-mythos-the-jag... (https://news.ycombinator.com/item?id=47732020)
> Anthropic’s cautious roll-out was justified. The problem with publicly releasing models, however, is that guardrails can be jailbroken, and apparently that is exactly what happened shortly after the release
The future is unevenly distributed. Anthropic, and Amodie in particular, seem to be of the mind they can control a bit of the unknown using words. They are likely being guided by the very product they built. *AI CAN MAKE MISTAKES
That Project Glasswing bullshit reeks of it. Corporations have take control of our attention, our Internet, and now our thinking.
I say it's high time to take it back.
As I understand it, ITAR regulations for export controls have just been applied to any form of Mythos. These are overseen by U.S. Departments of State and Commerce, and forbid foreign nationals from access to any form of Mythos, either within or outside the U.S.
Only U.S. citizens and immigrants that are holders of a "green card" may now access Mythos.
It appears that Anthropic does not have internal controls to implement these restrictions in any form, so the only option was to shut Mythos down.
Penalties for ITAR violation can reach ten years in prison and a million dollars per violation. (I can post a link to those details if there is any interest.)
As long as Anthropic is a U.S. company, there is no escaping this.
https://fortune.com/2026/06/14/how-a-warning-from-amazon-led...
That might be one of the most important points in the post. Very troubling.
It's questionable whether the current government can even unite the talent required for this project. Seizing it might just push all the talent to Europe or China.
The idea of open-sourcing something that falls into the "national security" category is clearly a non-starter unless there's more powerful, classified models that can outmatch them.
I think Anthropic has clearly demonstrated the most responsibility here: they've been crying for regulations, they were careful about Project Glasswing, and they've got comically over-sensitive filters around numerous topics.
if they had more success on alignment and safety research then I don't think the cludgy filters would be necessary
1) It’s safe to assume the US would do its best to prevent it, and even if Anthropic was successful in exfiltrating their data, code, models, and people, I’d imagine the US would immediately block all US companies from working with them. So they’d be blocked from their own US-based compute, plus Google, Amazon, Microsoft, xAI, Meta, etc.
2) Where would they go? China maybe, but as far as we can tell it doesn’t have sufficient compute for Anthropic’s level of need. The EU likely as or more restrictive in different ways to the US - the EU is hardly buzzing with AI innovation. Some Middle Eastern countries might have the money, energy, and interest in carving out such a position, but no compute. Plus I’d imagine the US would act directly against any country or region receiving them, economic or otherwise.
3) Then, as said elsewhere, the US would block GPU sales to wherever they found a safe haven, preventing the buildup of the compute they’d need to continue.
The EU options are not even close to what CF can do
That's not the problem.
The US government can export ban GPUs like they do now to more countries if needed. Even if the infrastructure exists, the GPUs won't.
This too, will end up being a good thing for them. The ban will end up getting lifted due to some "amazing deal" in the coming weeks and Anthropic will now have the "Trump tried to ban them, so they MUST have the most advanced AI model in the world!" stamp of approval just before IPO.
All this stuff is pro wrestling kayfabe.
Do they have it or do they just sell it?
Not affiliated with the bench in any way, but I think it surfaces important differences between the behavior of the models from different labs.
TLDR: The benchmark is measuring pushback in response to nonsensical requests and questions, as opposed to going with it and hallucinating a nonsensical answer.
[0]: https://petergpt.github.io/bullshit-benchmark/viewer/index.v...
I always thought safety was interesting in and of itself, but for some reason HN doesn’t have many people from the safety side of tech in conversation.
Tech isn’t a niche hobby anymore; Billions of people are impacted by the decisions of a few firms.
My grandfathers android had 3 different messaging apps installed, somehow. AI is enabling new forms of fraud at a time when we still haven't solved the old ones.
And this is all in the first world, move your coordinates to the developing world? We had human trafficking to get educated English speakers into call centers in Laos/Cambodia to defraud first world inhabitants of their money.
We aren’t in the early days of tech anymore, and the kind of scale that we have enabled comes with it a certain cost. We can choose to ignore them, or to understand them, but we will feel their impacts all the same.
I really dislike this belief (that has at least been expressed here) by some that X is okay because they-really-believe-it. This has a real Road to Hell stank on it.
It is incredibly convenient when your predictions or supposed beliefs go south. Well, we really believed that we were doing it for the betterment of human kind. And we really believed that X was an existential threat that was inevitable in which case we had to step up and do it because we we the only good guy ideologues. So sorry but not sorry.
I also don’t care if commenters know rank-and-file on the inside that “really believe it” as well. Not for one second.
So... what, you just don't trust anyone good? Would it be better to pull in a health insurance CEO? They're happy to watch people die for profits, no concerns at all about them pulling a "greater good" card because they're in it for entirely selfish reasons.
> So... what, you just don't trust anyone good?
The baseline here is apparently that they are good, I’m just supposed to trust and shut up?
Anthropic believes they have the responsibility to guard their tools from mis-use. That is all. They are not trying to "control" anything or anyone. They do however decide what they think is mis-use.
This only just shows how strong Mythos/Fable will be, once released to the public.
I'm guessing about 0.5 year till public.
Textbook retaliation for not letting them use an abliterated version of Claude in weapons systems.
This effectively renders any US closed model useless for any foreign company. Could happen to OpenAI, Google, etc. Too much of a risk to implement something that can be yanked out because the company didn’t behave the way they want.
Looks like it’s time for Kimi, Z, Deepseek to take the front row. They’ll catch up in a few months anyway. Kimi code 2.6 is crazy good
"When you further combine this realization with the company’s pronouncements about AI’s ability to conduct all economic activity, you realize that Anthropic’s leadership effectively wants to have power over everything and everyone."
This is fearful stuff on all sides, and none of the people involved might realistically be able to navigate the danger.
Reminds me of the RISC-V Foundation → RISC-V International move to Switzerland. Around the time some dumbass Republicans tried to impose export restrictions on a set of open, world-wide used specifications.
Pandora's box has been opened, and there's no closing it. Capable AI models will be everywhere.
The signal is clear enough though for the next Anthropic..
Europe has extradition treaties, so the U.S. can force anyone in Europe back to the U.S. for criminal indictment who demonstrates inappropriate possession of this technology.
Compared to their own API pricing, they're.
So either the subscription is a subsidy, or their API pricing is extremely... Overpriced.
Also I'd love to get the $200 sub at work. We get $500 in Copilot credits/month and it's much less then the $200 anthropic sub if I only use opus
It's also a lot better to get the limit staggered in 5h intervals for a dayjob
GPT refused to do so (citing that it's illegal even though I own the games). Deepseek did a wonderful job for 7 cents.
At work I use Opus because, why not? But I could easily switch to a less capable model if needed.
BTW, I also use DeepSeek v4 Flash very frequently: fast and so cheap it is almost free.
That seems to be the argument of Dario, Sam et. al., but I'm not ready to believe it. Time will tell, but this can be a marathon and Anthropic and OpenAI is in getting ready to sprint the last lap of the first mile.
The intuition is that distillation exploits not only the "right" answer but the relationship between answers (what's the second most right answer? the third? etc).
You can read it all over HN. It's about weakening American influence and building Eurocentric economies and influence. And exercising the same level of choice that Americans prefer as well. Americans also want to escape Google, Microsoft and Apple and more. They've all been caught investing too heavily in government influence and thought control (aka marketing).
And on the other side of that, an American company that deprives the US of AI for defense, is defacto weakening American defense because competition militaries will gain a technological edge by simply taking control of AI companies in their country which the US hasn't done (yet).
There are very valid arguments on both sides, I think.
Can you demonstrate beyond any reasonable doubt that the model weights have been transferred? No. Will the EU judges move to extradite said individuals (and many are EU citizens)? Also no, especially in the face of spurious accusations. And even if they were open to, you can stonewall everything and you will probably outlast any US administration pursuing that.
the next model with a gap to mythos as mythos is to opus will be controlled technology from the get-go. the one after it may be top secret.
By that logic, anybody who values safety has a god complex? It’s absurd…
"What this degradation represented was both the capability and willingness of Anthropic to silently alter its models to achieve its policy preferences. In other words, Anthropic willfully validated some of its critics’ worst fears in terms of being a supply chain risk."
IOW I don't think he thinks in the same categories as most folks here.
But I also think they exist in a sort of un-designed corporate narcissism, which is a common trait in bubble economies — I am not judging them particularly severely.
Netscape under Clark and Andreessen and Sun under McNealy both fell into corporate narcissism: the belief that only they really mattered, that they were chosen, and that the world needed to rearrange itself to just let them shine. They arguably let themselves get played by Oracle (a corporate psychopath) and others as a result.
OpenAI's position is profoundly corporate-narcissistic: all we need is all the money in the economy and not to have to do anything upsetting like think about turning a profit for the next four years. Like rich kids. It would be nice if you believed we were so important that we should get an enormous stipend for just being us.
Anthropic's position is: we think we're so unique and ominous that government needs to make us both essential and terrifying. We have to exist otherwise worse people will.
Both narcissistic positions.
it has to be, because the other way around - the government taking over parts or the whole thing - is inevitable if the trend holds.
in general I agree people should be reading a lot more sci-fi nowadays than they used to.
The safety side of tech is a PTSD inducing shit show. Governments are more than happy to champion age verification laws, because parents, around the world, are clamoring for anything to pump the breaks on the social media experiment.
Society outside of HN is quite tired of Tech, and I despair of figuring out a way to make this clear to the commentariat.
As someone on the "safety side of tech", social media is being exploited to increase surveillance and government control precisely because its actual social influence is heavily on the wane, and capital is happy to sacrifice what's left to increase the profits of the expanding public/private tech surveillance industry (with "protect the children" controls on social media like age verification being the usual backdoor route it always is).
Society may be growing tired of Tech, but governments aren't, and in fact they're heavily expanding their back channel reliance on not-traditionally-military Tech as an extension of their Defense spending.
Social media was being exploited from inception. Palantir had sales documents for sock puppet management software back in the PHP era.
I don’t disagree that Government is interested in tech, but I will push back on the dismissal of child safety that is inherent in your comment, intended or not.
For all that some people in the firm may have tried to do the right thing, Social media firms have created bad outcomes for children, and executives were briefed on the harms they were going to cause.
This is the dismissal that concerns me, because it ends up miscalculating the level of anger and unhappiness amongst the voting populace, and therefore the political will to pass regulation to reign tech in.
I don't think anyone in tech is really truly engaging with how quickly the shine has come off the tech industry. Except maybe Apple, who even so still have some work to do.
The yoke of silicon valley is feeling heavy. People might just throw it off.