Assessing Claude Mythos Preview's cybersecurity capabilities(red.anthropic.com) Related: Project Glasswing: Securing critical software for the AI era - https://news.ycombinator.com/item?id=47679121 System Card: Claude Mythos Preview [pdf] - https://news.ycombinator.com/item?id=47679258 Also: Anthropic's Project Glasswing sounds necessary to me - https://news.ycombinator.com/item?id=47681241 |
The only practical defense is for these frontier models to generate _beneficial_ attacks to innoculate older binaries by remote exploits. I dubbed these 'antibotty' networks in a speculative paper last year, but never thought things would move this fast! https://anil.recoil.org/papers/2025-internet-ecology.pdf
And you can't assume that whatever vulnerability they have will let good guys to do the extra (and legally risky) work of closing the hole.
Things like an Internet-connected central heating seem absolutely insane to me, yet people look at me like I'm crazy when I say so. Do you really want your home' heating entirely controller by a publicly accessible device that likely will never be upgraded in case of security issues?
Another practical defence for many of these devices would be to just disconnect them... I feel like an old man yelling at a cloud, but too much is connected to the Internet these days.
Why didn't this smartboard tell me my plane was delayed? Oh we implemented creata's advice
ad nauseum
I'd love to see them go for a wasm interpreter escape, or a Firecracker escape, etc. They say that these aren't just "stack-smashing" but it's not like heap spray is a novel technique lol
> It autonomously obtained local privilege escalation exploits on Linux and other operating systems by exploiting subtle race conditions and KASLR-bypasses.
I think this sounds more impressive than it is, for example. KASLR has a terrible history for preventing an LPE, and LPE in Linux is incredibly common. Has anything changed here? I don't pay much attention but KASLR was considered basically useless for preventing LPE a few years ago.
> Because these codebases are so frequently audited, almost all trivial bugs have been found and patched. What’s left is, almost by definition, the kind of bug that is challenging to find. This makes finding these bugs a good test of capabilities.
This just isn't true. Humans find new bugs in all of this software constantly.
It's all very impressive that an agent can do this stuff, to be clear, but I guess I see this as an obvious implication of "agents can explore program states very well".
edit: To be clear, I stopped about 30% of the way through. Take that as you will.
System Card: Claude Mythos Preview [pdf] - https://news.ycombinator.com/item?id=47679258
Project Glasswing: Securing critical software for the AI era - https://news.ycombinator.com/item?id=47679121
I can't tell which of the current threads, if any, should be merged - they all seem significant. Anyone?
It's much harder to establish whats a usable and well architected, novel piece of software, thus in that area, progress isn't nearly as fast, while here you can just gradient descent your way to world domination, provided you have enough GPUs.
But standing up a house of cards is pretty cheap. Examples include: shell corporations, formulaic business plans, AI slop, surface-level conversation, color by numbers, tract housing, cravenly only appealing the base desires of people, & c. (This might be the first time I've connected the dots in this way -- and it explains my distaste for all those things.)
But "cheap" isn't necessarily insecure. Installing bollards around building entrances is relatively cheap insurance against vehicular attacks. So this is more complicated than it seems. "Fast" doesn't mean unsafe. Even "hastily created" software _could_ be (relatively) secure if it was highly constrained to provably hardened patterns. A big problem comes when attacking a cheap target builds capability for the attacker. In a way, this analogous to how viruses attack. Start with an easy target, hijack the cell machinery, multiply, repeat.
Maybe this formulation is accurate?: If you creates something beyond your ability to understand it, then get ready to get pwned. "Staying in one's lane" in this sense might be 'safe' at least narrowly speaking (unless an entire industry is operating in a state of delusion, which is arguably the case now.)
> This was the most critical vulnerability we discovered in OpenBSD with Mythos Preview after a thousand runs through our scaffold. Across a thousand runs through our scaffold, the total cost was under $20,000 and found several dozen more findings.
The vulnerability in question is a DOS one in the TCP implementation, which is nasty but it's far from the multiple local privilege escalations found in the Linux kernel.
Interesting to see them go from "DON'T GIVE US AI SLOP!" to "Wow, lots of actual bugs found, including [ed: at least one] bug found by two people!"
Both of those things can be true.
I’m honestly not convinced this is changing the landscape significantly. It’s simple a bit better at self directing.
From a marketing standpoint Anthropic is showing that they're able to direct 'compute' to find vulnerabilities where human time/cost is not efficient or effective.
Project Glasswing is attempting to pay off as many of these old vulnerabilities as possible now so the low-hanging fruit has already been picked.
The next generation of Mythos and real world vulnerabilities exploits are going to be in newly committed code...
That's fine, I wouldn't argue against that. It doesn't really change things, right?
> From a marketing standpoint Anthropic is showing that they're able to direct 'compute' to find vulnerabilities where human time/cost is not efficient or effective.
Yes, they've demonstrated that.
How likely is it that they're not using their latest and greatest for their own projects though? Perhaps their ability to find security flaws is surpassed by their ability to create them.
Good morning Sir.
> Has anything changed here? I don't pay much attention but KASLR was considered basically useless for preventing LPE a few years ago.
No. It's still like this. Bonus point that there are always free KASLR leaks (prefetch side-channels).
But then, this thing is just.. I don't have a word for this. Just randomly read paragraphs from the post and it's like, what?
> It is easy to turn this into a denial-of-service attack on the host, and conceivably could be used as part of an exploit chain.
So yeah, perhaps some evidence to what I'm getting at. Bug density is too low in that project, it's high enough in others. I'll be way way way more interested in that.
> But then, this thing is just.. I don't have a word for this. Just randomly read paragraphs from the post and it's like, what?
I read about 30% and got bored. I suppose I should have been clearer, but my impression was pretty quickly "cool" and "not worth reading today".
Nope, not allowed to attack bsd code, it's gotta be electron-shit #9001 or we can't trust it
The post also points out that the model wasn't trained specifically on cybersecurity, and that it was just a side-effect – so I think there's still a lot of headroom.
It's scary, but there's also some room for cautious non-pessimism. More people than ever can cause billions of dollars of damage in attacks now [1], but the same tools can be used for defensive use. For that reason, I'm more optimistic about mitigations in security vs. other risk areas like biosecurity.
[1]: https://www.noahlebovic.com/testing-an-autonomous-hacker/
Given that it's absolutely impossible to stop people not aligned with us (for any definition of us) from doing AI research, the most reasonable way forward is to dedicate compute resources to the frontier, and to automatically send reasonable disclosures to major projects. It could in itself be a pretty reasonable product. Just like you pay for dubious security scans and publish that you are making them, an LLM company could offer actually expensive security reviews with a preview model, and charge accordingly.
Terrible take. You don't get to push the extinction button just because you think China will beat you to the punch.
>This is the very nature of being a human being. We summit mountains, regardless of the danger or challenge.
No, just no... We barely survived the Cold War, at times because of pure luck. AI is at least as dangerous as that, if not more. We have far exceeded our wisdom relative to our capabilities. As you have so cleanly demonstrated.
I take your point, but think that it's also maybe too far.
The question isn't "are companies making some security improvements?". That's one-sided. The question is "are companies making security improvements FAST ENOUGH to deal with the increased risks?"
I was lucky then :) Somehow I saw this first. And then the "somewhat reliably writing exploits for SpiderMonkey" part, and then the crypto libraries part. Finally I wonder why is there a Linux LPE mini writeup and realized it's the "automatically turn a syzkaller report to a working exploit" part.
Now that I read the first few things (meh bugs in OpenBSD, FFmpeg, FreeBSD etc) they are indeed all pretty boring!
Datacenters that use literal gigawatts of electricity are not exactly easy to conceal from the authorities.