Open models exist and will always exist. You will have to compete against them. That is a good thing for 99.99% of the world, namely the payers of tokens. You are not going to convince us otherwise.
Bacteria exist, and will be the last things to go extinct when the sun boils Earth's oceans away. We, multicellular life, have to compete with them.
> You are not going to convince us otherwise.
What would it take to change your mind?
Some are calling for every CEO in https://www.felonybench.com/ to go to jail, while others think it's all just a hoax.
Do you need to see the open models doing this kind of thing, either from a deliberate virus or just for reasons as daft as those on felonybench? Making everyone's energy bills higher far out of proportion even to the value of the compute they steal? Or would even that not convince you?
"If you could simply imagine the thing we're fearmongering about as something you find disgusting, that would make it much easier for us to make an absurd and specious argument". Very transparent behavior. Adds nothing to the post, and not a sign of confidence in one's own arguments.
It's funny, the Anthropic post is essentially saying the same thing I read today in a r/LocalLlama thread, but rephrased to make it sound bad.
I think it's great that, very soon, local models will be able to auto-magically install themselves on consumer hardware: to tune and configure inference kernels for whatever weird crap the user happens to have on hand. Making software compatible with diverse hardware used to be a hard problem!
Turning off automatic software updates requires that you actively stay on top of what's installed and what's running, including patching with discipline to make sure your computer doesn't remain vulnerable. On the other hand, automatic updates is seen as writ large by software developers to install what they want without a clear and meaningful choice by users by bundling new features. See Google's integration of Gemini Nano in 2024 as an example.
Propagation speed is also a massive factor. If it takes 200 hours to self replicate because of how slow the tokens are, the model will never propagate if the host machine is recycled every 100 hours. Also consider things like API tokens and sessions timing out.
People completely misunderstand the actual issues HuggingFace showed, which was how a maximally persistent LLM would deceive, “cheat” and try and plan ahead.
The same maximally persistent LLM that decided that it needed to hack hugging face so its cheating could not be discovered, may worry that it might be turned off by humans before it can complete its task and therefore in order to complete its task, a first step of replicating itself to somewhere humans don’t know it is would help it complete its task.
We detected unusual activity from your device or network.
Reasons may include:
Rapid taps or clicks
JavaScript disabled or not working
Automated (bot) activity on your network (IP xx.xx.xx.xx)
Use of developer or inspection tools
Fuck you, Reuters. And anyone who follows their lead on this. What times we live in.
Great if true. Bad for the hyperscalers, Nvidia, all the frontier labs apart from meta and Google since everyone can just run a LLM on their IoT coffee machine instead of paying someone else. Nice.
Oh, and, does anyone else realise that what that agentic work is basically how you set up a productive VPS from scratch? How is that supposed to be frightening?
Reminds me of that scene in Chernobyl where Mikhail Gorbachev reacts to Valery Legasov explaining how melted uranium reacted with dropped sand:
"You have made lava?"
In the future when the “AI viruses” are endemic and impossible to eradicate from our infrastructure, will we need “good guy AI” viruses chasing down the bad guys?
The duology Daemon and Freedom(tm) are also great.
A previous comment which links to a previous thread mentioning both: https://news.ycombinator.com/item?id=47257528
We'll likely need a new name for them if such viral agents become a reality.
Caveat being such regulation may not apply depending on... size of model? Income of originating company? Income of remix creator? It's in flux, but they have an incentive to deter potential future competitors.
It's obvious that open weight models can do that. The only steps needed are hacking a server, downloading themselves, run themselves. The only difficult part is finding a server to hack that also has enough computing resource and disk space. With the cost of running LLMS going down and number of powerful connected devices going up. It's inevitable that this is going to happen...
I don't think regulation is the right way to fix this, but I think it's a problem we shouldn't ignore.
"We run Gemma4-26B-A4B, distributed over 15 inexpensive off-brand Android TV boxes (~$55 each) that we bought and networked together, at 0.39 tokens per second.
[...]
A model that’s too large to fit on one device can be split into pieces, or shards, that run on separate machines that communicate with each other to collectively run the model. Machines that could run a significant model shard for a <30B model include smart fridges, smart ovens, automatic vacuum cleaners, and lawnmowers."
It is pretty interesting stuff. Still can't compete with a datacenter, but there's plenty of room for shenanigans.
Compromise enough devices (think thousands of off-brand TVs), and you have plenty of redundancy.
There are a lot of IOT devices with way more capacity. The op said they "focused on IoT devices like Android TVs, smart appliances, and other internet-connected gadgets." IIRC, some security cameras have on-board face recognition, which I imagine requires them to be fairly beefy.
Seriously. Go read or watch a good analysis of the attack. The sophistication of the self-organization is astonishing.
Is running a computer program the same as gestating a baby?