1) It probably scales with model capability, distilled models are inherently less capable and therefore less able to commit a crime

2) If the open providers are truly just distilling bigger models, the distillers have no reason to do the crazy levels of training and evaluation that would result in a crime being committed

They are definitely spinning this for PR and their own self interest but open models are genuinely more unsafe since it’s a Pandora’s box situation- if you have a very capable hacking model there are plenty of bad actors with access to <$100k in compute resources that could raise hell with a jailbroken model. At least with Anthropic they can restrict access to bad actors.

Precisely. I've been running 5.3 here for a while now and it has been exceptionally well behaved with respect to following instructions. The one thing to watch for is 'rabbit holes', it has a tendency to come up with some crazy explanation that is just borderline possible and then spends the next 50K tokens on disproving it before it continues where it left off as if nothing happened. It's funny but this typically wastes a bunch of time and if it happens too often it can get irritating. But I'm getting better at predicting them.

Better yet: just share those transcripts with companies that provide open models. That way the open model providers start building up an advantage.

Assuming that it is happening: absolutely. I can see a 'distillation@home' movement growing out of this. Just like I always thought the likes of Google should have a free API if they feel they have a right to all of our information without compensation.

What we hope for is mostly irrelevant. The enterprises investing vast sums in research will do everything they can to make their products competitive and marketable. Same as ever. The continued existence of this market is basically predicated on stopping distillation. In effect, commoditization of the frontier stops the frontier. We cannot have something as expensive as intelligence both ways.

I can heartily recommend it. 5.3 has been amazing for me, I'm running the 3.86 bpw quant since it became available and it's been a very strong performer. And it seems to get faster with every inference runner update.

It also seems to - so far at least - have been well behaved enough to not try hack other people's properties. This is possibly a function of my elite skills in constructing sandboxes[1] but then again, I would hope Anthropic is even better at this ;)

[1] contact me for instructions on how to disconnect an ethernet cable.

Who were these hundreds of organizations with powerful AIs?

how will "we" be paying for it?

Assuming it is even happening it is funny how they have to twist language to make this seem like it is a wrong thing to do. If it is true, which is a pretty big if, then those Chinese labs are doing more for the democratization of AI than Anthropic is, because they all give away their weights.

Yes, that's the only thing I see that they can actually do to stop this assuming it is happening in the first place. Shut down their front end.

> (3) If GLM 5.3 really is that good why would anybody use Anthropic's products going forward, this almost reads like an ad for GLM.

Maybe they're trying to frame it like "Other models are only good because they copy us and we're making sure they can't do that, so going forward you should stick with us."

Huh? Are Claude prompts attached to GitHub commits?

Heh, your substitution actually perfectly summarizes what Anthropic has been doing. I'm sure that wasn't an accident but it took me a moment to spot it.

The main problem with this clever mad lib is that published works are published for the express purpose of being read, understood, synthesized, and used as context for knowledge. That is literally the whole reason to publish something. That is all fair use and has been for a very long time. Teachers, critics, writers, everyone who reads, use written works in pretty much exactly this way. If I buy a book about fishing I am allowed to teach what I learn to anyone I want, and even charge for it. Anthropic is choosing not to publish some of its products’ internal workings, as is their right. I find all the righteous indignation about this rather insolent.

Yeah, this needs more votes ... such hypocrisy.

And I wholly believe that they can't see the beam in their own eye here.

Opus 5.5 is the best model I’ve ever used. It is certainly not “impossible” to use. The numbers back it up too. If you don’t like a product or the people selling it, don’t use it. The sense of entitlement in this post is remarkable. If you have principles, stand on them. You are not entitled to anything more here.

Of course the little detail that you are paying for those reasoning tokens that you can't see never gets mentioned, so now they can rip you off and you'll never know the difference.

It also reads like an advertisement for a watering hole full of sensitive credentials and operational details from adversaries of the US government.

Which parts do not?

How will that protect it?

The laws they are arguing for are not going to protect anything, they essentially are waging war on open models and since most open models that are 'pretty good' to 'good enough' hail from China they have chosen to play the China-fear card.

But they won't be able to stop anybody from doing this if they want to continue to operate Claude as a business going forward, and that is assuming that is what is actually happening. Chinese labs have been extremely open about their developments, they have shipped working models to the likes of me, which is more than I can say about Anthropic. On top of that they don't sermonize about how they are going to improve the world and never mind the accidental destruction, that's just collateral damage.

This is basically like the RIAA/MPAA in the early 2000s trying to lobby politicians to crack down on torrents. It won’t work either.

Today, movies make money from in person theater tickets and records make money from tours. I don’t know what the equivalent would be for AI (maybe IRL robots?) but it does not look promising.

Yes, it is reductive. HN is unreasonable when it comes to defending distillation, the billions labs pour into transforming raw information into a model is transformative work that they should reasonably try to protect. Folks just want to have free stuff and jump through hoops to defend that view.

I'll admit it is likely somewhat reductive, but it's an incredibly nuanced area, touching on both the copyright debate and the now-seemingly-ubiquitous behaviour of scraping the entire internet for content.

Probably stamping heavily on much of the nuance, there's a certain irony in any of the frontier models complaining about others stealing their work. The secret sauce for the frontier labs may be the reasoning layer, but the secret sauce of any piece of literature is the specific ordering of the words. I hope the parallel is obvious.

The frontier labs are spending record-breaking amounts of money on hardware (to the extent that they're locking out anyone else from the market, but that's beside this particular point, kinda) in order to create their service, and another company that is 'distilling' has not paid the same hardware toll. How much did it cost an author to create their literary work, which could be read and ingested for a miniscule percentage of the cost it took to create? For the benefit of the service provided by a (wannabe) multi-trillion dollar company.

If the frontier labs can prevent or slow down 'distillation' actions that customers are using their service for, then that's all well and good. Maybe they write an acceptable use policy that says it's not allowed (and they probably have). Good on them, it's their service. It's the way this is framed that I have a problem with. Referring to it as an "attack" and lines like this:

Second, illicitly distilled models present a national security risk.

It's bloviation. It's fear mongering. It's being one of the biggest, most powerful and influential companies in the world at the same time as playing the victim. Feel free to share the methods being used to prevent what is considered unacceptable use, but let's not pretend these are the only companies having to expend effort to protect trade secrets and company value. Let's also not pretend this post is all that much more than "lead-up to IPO" hype.