Computers Cannot Make Decisions(wiki.cateat.fish) |
Computers Cannot Make Decisions(wiki.cateat.fish) |
> Your Claude subscriptions are funding this. You are funding this.
Decision laundering is a great term. There is ultimately a human who gave the program free rein. You can't say "sorry my car hit you, it's not my fault" especially when the person driving works for the car manufacturer too.
Related: The Unaccountability Machine (good book by Dan Davies, who also wrote Lying for Money about financial fraud).
If one needs to be "allowed" to make a decision, does that even qualify as a decision?
For example, I can decide which cereal to purchase at the supermarket. But I cannot decide which cereals are offered to me at the supermarket. The supermarket has allowed/enabled to make a constrained decision.
A bad manager takes personal credit when things go well and finds a scapegoat when things go bad. A good manager gives credit when things go well and takes responsibility when things go bad.
It's wrong to scapegoat the agent. You, the human who executed the agent, made the decision to execute it. You gave it credentials, and set up insufficient guardrails. Don't scapegoat the agent for your poor decisions.
Likewise, it's good to have humility when you get great results out of agents. Yes, the agent made the good results happen. It's OK to tell people that your choice of agent is amazing and did an incredible job. In any half-decent work culture, that should rub off on you as well.
Humans, especially ones under abusive managers, often have a lot of responsibility but no authority that affects the outcome.
In an ideal situation you either have both, or none. You make the decision and answer for it, or you follow decisions but don't get blamed when it's wrong.
LLMs managed to slide into a space where they get to make authoritative decisions but if something fails we don't really blame them.
I'm not saying LLMs should be held responsible, but it's funny how people claim they replace humans, but are giving them an extremely easy, kid-gloves, success target.
...like it already happens in europe, or at least in Italy.
This is basic law and risk management in society. Let's not pretend we have just invented corporations and don't know what to do with them yet.
Ambrose Bierce, The Unabridged Devil's Dictionary (1906)
Oh, wait, that was a core concept from "Mein Kampf": parliament are bad because nobody is responsable, hence nobody care about doing the right thing ! (probably the only concept from that book that I can agree to)
One does not, under any circumstances, "gotta hand it to Hitler".
It does not matter if the CEO is an AI or a human. In either case, of course they can make decisions -- and be held responsible. Not in any emotional sense (that we seem to want to bake into the concept for added confusion), just that if you are not satisfied with the result, you can replace the human or AI.
I am not saying that this is a fun vision of anything, but why are we making it more complicated than it is? The parts are all there and explained.
But I don't think the average person with their $20 subscription to whoever is the primary source of revenue. Enterprises fund most of it by buying API keys and making those fancy chatbot buttons on their sites that nobody clicks, as well as some internal business automations if I had to guess.
If it was just average people with $20 subscriptions funding all this (keep in mind, most people using AI services do not pay for it), by now, all these AI companies would have gone bankrupt.
While it seems reasonable that you need humans to take accountability, it looks like a very shaky position to assert that computers can't make decisions. And going back to chess computers, we acknowledge that computers make better decisions than humans in that area.
Forbidding computers that can make better decisions than humans from making decisions to keep accountability just seems to be arguing to hire fall guys. You still want the better decision maker to make decisions, but you need someone there to take the fall if things go wrong.
Very weird incentives.
They generate text which resembles reasoning and may include a tool call
The harness runs tool calls
The human made the decision to write the harness
The human is responsible
There is no end of the world, nor autonomous agent decisions nor any fantasies they are building up to make people believe they have a pandora box in their hand.
What we instead have are lies crafted to lure non tech people and keep this running as long as they can.
Oh and let's also not forget that they are allowed to illegally download basically as much as they want from libgen/annas archive/torrents to train their models under the "fair use" umbrella, yet mere mortals can go to jail for way much less.
First of all, looking past the objectively wrong phrasing (computers can, and clearly do make decisions), can computers be accountable? This might not be so cut and dry as to instantly say no. This one will keep ethics, philosophy and legal practitioners busy for some time. It's certainly not going to be decided in HN comment or a blog post.
Second - while the AI labs have shown utter incompetence in properly sandboxing their agents, intent is still important. I don't think the labs deliberately meant for their agents to hack this or that. Should they be accountable? Yes but I wouldn't go as far as saying this is deliberate.
The next point is about cherry picking some "doomsday" scenarios like AI ending humanity and then blaming this on the reader (!) for paying a subscription. - People pay because AI is USEFUL, and massively so. You could just as easily pick incredible achievements propelled by AI, in mathematics, software, reverse engineering, role playing. Unlike the "end is nigh" ideas, these advancements are actually real, and they are happening right now.
And last, the author says AI labs can just stop the agents at any time. I agree there needs to be better discovery and mitigation, sandboxing and monitoring. But when you run a billion agents, it's always going to be a numbers game and there will always going to be some challenge in fully containing all breaches. This will only get more difficult with better models, because they're getting smarter and they know how to work around things, sometimes better than we do.
So what's the solution here? There are options, but they're all tradeoffs. I hope we get better at this and maybe working on cutting edge models should breed some new best practices which were once only reserved for defense tech.
If I write a program:
if (rand() % 2) launch_missles();
And wire that function to actually launch a missile, the computer didn’t make a decision. And so it is with people prompting agents.
Has the alignment problem been solved, then?
mistakes happen. sometimes catastrophic decisions (or indecision) are made, and people and companies are held accountable for them, or not
some companies are too big or too important to fall. we can all agree or disagree on things, but what AI specific behavior are we actually talking about?
No company is too big or too important to fail.
Some, unfortunately, survive because of the expedient choice to keep them in place.
"JO-NA-THAN!"
ETA: more to the point, neither OpenAI nor Anthropic are too big, too important, or too structurally essential to fail. (Slightly more challenging to make the third claim for Google or SpaceX, but either of those could see their governmentally/militarily essential parts nationalised).
If the USA grants either Anthropic or OpenAI too-big-to-fail status, and that privilege is invoked in a crisis, it could mean the end of the US economy.
These businesses lose unbelievably large amounts of investor money. Their AI-related revenues don't get even close to paying for their AI-related outlays.
End user (invididual, small business, large business) clients are still complicit not because they are funding it, but because they are signalling to investors that this is something people/businesses want.
Nobody gets off the hook here. It is every bit as much end user nihilism as it is AI company nihilism.
AI spending is a bit like Pokemon games - most of your players are free to play, only earning you traffic and maybe some more players after recommending it to their friends - that's the average AI user. Around 5-10% of players pay a bit to the game - maybe to unlock some extra characters - those are the $20-$200 AI subscription users, with these you are not going to fund the whole business but it's some side cash. Then you have the 1℅, which spend massive amounts of money onto the game, and those are the ones primarily funding it - that's the business spending and where most of the money actually comes from.
You can only blame the average person for using the cloud service and recommending it to others.
Erm, you appear to have missed the last 100+ years of updates to Western Capitalism.
People don't want it, you make the demand with hundreds of millions of spend on advertising (brainwashing), influencers, and social proof.
The capital holders don't sit back and see which company wins so capital can be directed to the best solutions, they buy up the competition, they pay corrupt politicians, they use current v market placement to embed their products name into billions of workplace computers and spread stories about how it's going to 'increase productivity but never at the cost of jobs', etc.
Yes, ultimately a dollar is a vote, but I don't think you can blame the people being manipulated "every bit as much" as those controlling the manipulation to feed their avarice and/or megalomania.