Has Mythos just broken the deal that kept the internet safe?(martinalderson.com) |
Has Mythos just broken the deal that kept the internet safe?(martinalderson.com) |
The initial batch of publicly disclosed vulnerabilities by Mythos demonstrates that perfectly. None of the bugs themselves are especially interesting or complex, in my opinion. They were found by applying effort to a very large amount of code which included under-scrutinized areas, where bugs hid. Yes, even in projects like Linux and OpenBSD there are many pieces of code that aren't that properly vetted, because of the finite amount of developer/researcher time allotted.
The fact that this effort is much cheaper does indeed change things. But really strong sandboxing solutions, such as gvisor or firecracker, do a really good job of having very little attack surface, all of which is heavily scrutinized.
Until we see more of the bugs that were found, it remains to be seen whether or not the post's premise about sandboxes is correct.
Working in this space, I'm worried the future is even more reactive than today. Today I can get teams to review for security the architecture before implementing, and to review the implementation for security before shipping.
As teams move (more specifically, are forced to move) to vibe coding the whole thing, nobody knows what the design is or what the implementation looks like. Vibe all the way because the CEO says so or you're fired. This means the only place to catch vulnerabilities becomes after the fact, which is usually too late.
Seems something of a unusual take on the state of the world
For example a lot more people would sue eachother for petty things if it suddenly became very easy and cost efficiant. Its not, so they dont.
Another example of AI doing this exact type of thing in another realm: In the past convincing someone you were somebody they should give money to for a scam was very possible to do, but also difficult and not very cost efficiant. You could try to impersonate someone's daughter or a police officer, but it took a lot of effort to get it right.
Now, with voice mimicking ai, deepfakes, social media to mine for personal info, etc its not as difficult and so, very likely, its becoming a bigger problem than it was.
Are folks going to actually go back and fix things that were only secure because they were or buried in layers of obfuscation and obscurity?
Probably not. And that’s the real cyber security risk. Short term profit always wins.
> The amount of energy needed to refute bullshit is an order of magnitude bigger than that needed to produce it.
Now the energy needed to secure against exploits is orders of magnitude bigger than the effort needed to secure it.
The combination of deep expertise + infinite patience of the LLM meeting the vastly increasing surface of software has a certain apocalyptic chaos gods ruin to it all, just as well known bias for mistruth to unfairly propogate itself bedevils this good planet.
At most, Mythos has reminded us that this "deal" is subject to frequent cycles of being compromised-and-patched.
From time to time, I have run browsers configured for opt-in javascript (eg, umatrix), but man it's a lot of work to live that way.
Heaps of prior examples of buffer overruns processing image formats, or the FFmpeg mentioned by Anthropic's Mythos article.
Admittedly JavaScript is often used as part of a chain to simplify attack (e.g. repeating heap spray to attack a use-after-free reliably).
The interesting part is that Anthropic have talked about automating finding vulnerabilities and also automating exploitation of those vulnerabilities.
Why are AI people so dramatic? Ok, there is yet another JS sandbox escape - not the first one, not the last one. It will be patched, and the bar will be raised for a bit... at least until the next exploit is found.
If anything, AI will make _weaponized_ exploits less likely. Before, one had to find a talented person, and get pretty lucky too. If this AI is as good as promised, you can have dependabot-style exploit finder running 24/7 for the 1/10th cost of a single FTE. If it's really that good, I'd expect that all browser authors adopt those into their development process.
Dario is trying to scare you to buying into his IPO and you're over-estimating the capability of Mythos...because he said so? With no independent reviews on the research and with many security researchers and experts accusing them of blatant scaremongering.
This is Anthropic's latest attempt to frame local models and to get them banned as they stand to be a threat against their business model.
The double irony here is that the many projects being rewritten in Rust introduces a ton of new bugs simply due to the rewriting process, on top of using up a lot of dev effort
Stated differently: right now black hat hacking is a valuable skill that can be turned into money easily. Once everyone can do it the incentives shift and the black hats will disappear. And that leaves the next most incentivized group in control of the market, who are presumably the software vendors.
Basically Microsoft and Google and company used to have to pay bug bounties and pray. Now it's practical just to throw a few million dollars at Anthropic instead.
No, they launched a card with that capability written on.
when shareholders are basically the same, and this companies have a legal obligation to fulfill their interests...is it a conspiracy? shareholders certainly conspire to achieve their goals, smarty
BTW: Mythos is not new. OpenAI literally released a press release 1 month ago talking about GPT 5.4's redteaming features being so powerful they require ID verification to use it, and will use heuristics to downgrade you if you look like you're doing something shady. I guess everyone's got a short-term memory, or Anthropic's PR is so good that people genuinely don't understand that OpenAI's models are superior to Anthropic's.
That is a provocative statement that would be especially interesting if you were to add some supporting evidence.
"BTW: Mythos is not new. OpenAI literally released a press release 1 month ago " these two sentences make no semantic sense together
Not you. EVERYONE doing ANY kind of software will have to, because else attacker can just pick and choose targets to point their exploit-bot
That's not at all clear. JS escape exploits have high value in our current Internet so there's going to be a lot of prior art. It's not surprising at all that this is what their model found and it's not a statistic that immediately suggest it has any broader implications.
Mythos seems much, much more creative and self directed, but I’m not yet convinced the core capabilities are significantly higher than what’s possible today.
The full price of finding the vulnerabilities was also something like $20k. That’s a price point that brings a skilled professional in to accomplish the same task.
It’s just fascinating to see how AI’s accomplishments are being systematically downplayed. I guess when an AI proves that P!=NP, I’m going to read on this forum “so what, mathematicians prove conjectures all the time, and also, we pretty much always knew this was true anyway”.
But yeah, if their model can reliably write an exploit for novel bugs (starting from a crash, not a vulnerable line of code) then it's very significant. I guess we'll see, right?
edit: Actually the original post IS dramatic: "Has Mythos just broken the deal that kept the internet safe? For nearly 20 years the deal has been simple: you click a link, arbitrary code runs on your device, and a stack of sandboxes keeps that code from doing anything nasty". Browser exploits have existed before, and this capability helps defenders as much as it helps attackers, it's not like JS is going anywhere.
What would be the practical impacts of this discovery?
Codegen for many companies is much less continuous. Security is always on, and always a motivator.
So yeah, dependabot, but the richest actors will have the best bits and they probably won’t share the ones they can find that nobody else’s models can
Presumably we would not give the AI models to the "good guys" because then they would also find and patch these vulnerabilities?
Scepticism means staying wary and keeping one's mind open, and not closing your eyes to a new reality.
We already operate on the assumption that P ≠ NP, so little would change if that were proved.
Would it become crackable, or just theoretically crackable?
E.g. it's one thing to show it's possible to fly to Mars, it's another thing to actually do it.
* It's possible - very likely even - that even if somehow P=NP, the fastest algorithm for any NP problem turns out to be something like n^1000, which is technically P, but not practical in any way.
* The proof may not be constructive, so we may just know that P=NP but it won't help us actually create an algorithm in P (nitpick: technically if P=NP there's a construction to create an algorithm that solves any NP problem in P time, but it's extremely slow - for example it involves iterating over all possible programs).