Who manages the agents?(off-policy.com) |
Who manages the agents?(off-policy.com) |
Many indistries are changing, but in most cases the new tools will be more akin to cars that still need drivers, rather than robots who take over the whole job. Yes, jobs might be lost, or shifted to others, but it's not like suddenly 90% of people will have nothing to do. There were similar shifts in the past with new technologies, and we made it past them.
Some of these people have lost their damn minds.
People building an agent framework that will struggle to correctly infer that my appointment at a hospital will require additional travel time when organising my calendar for me waxing lyrical about the future of the humn race is chaotic behavior.
Th Wright brothers would have had no credibility discussing what ATC protocols should be, and they, at least, actually did something credible.
The problem is they really did something credible, and this will let them concentrate huge amounts of power (not hold it, but collude with those who will hold it) and define your future. The longer you dismiss it, the bleaker it will be.
i try to think of this whenever i am going into ai psychosis.
I guess my time zone wasn’t in the context? So it just hallucinates the wrong coastal time zone twice when I’m not in either one. But who knows where exactly it messed up because it could have just picked a random hour and a would have had the same outcome.
I’ve only used Alexa a half dozen times since the release of Alexa+ but it has been confidently incorrect about 100% of the queries.
"ultimately accountable for the success or failure of a specific project, initiative, or activity"
I think that role should be reserved for a human, who can then use all the agents they like but has to take accountability for what is ultimately delivered.
A bit dramatic for effect but true.
what's developing is more about scape goats than anything rational like responsibility.
The middle management in companies is one of the worst inventions ever. I think baboons have better middle management structure than us.
Might as well replace all that.
I'm not saying corporations execute on this flawlessly, just any time when I wonder what I would've done myself, I end up with a similar structure (maybe a bit flatter)...
"The famous primatologist Robert Sapolsky has spent decades studying baboons and has described how status, alliances, conflict resolution, and coalition-building resemble politics inside human organizations"
Baboons are a good example for studying mammal social management strategies, which humans also do.
They are organized by dominance hierrarchy, like humans, but baboons have a distributed leadership.
I think we could learn a lot from baboons when it comes to management
You know, as an IC, if I get get the level of introspection you can get from building distributed systems, and that with reasoning/thinking traces from LLMs, I think I'd prefer the entire level of middle management made out of LLMs rather than humans. It'd be helpful to be able to see exactly where their logic suddenly took a skip out the window.
LLMs will not be centralised or restrained to any 'clergy', the rabbit is already out of the hat, and open-weights models exist and are widely used. Probably not as good as the latest Sol and Fable but 95% there.
Codex and Claude Code without a doubt have very good models behind them. But they also have really good harnesses built around them. An LLM is only a brain stuck in a cranium in the dark. It can generate endless code/prose, but it can't walk or see on its own, it needs additional tools. If you read any of the local LLM subreddits you will notice people mentioning again and again that the harness/tool-use/template-tweaking makes all the difference on how a model behaves/on how smart it is perceived.
Some folks are already using Qwen models for their daily work. Maybe it can't work in a hands-off/one-shot fashion like the frontier models, but they can help tremendously if you already have some domain knowledge.
People are excited about local LLMs and it's not going away any time soon.
> Your agents need to be sovereign. Your company must own and control the agents’ identities, permissions, memory, skills, artifacts, and audit trails. Those assets must be portable, governable, and inaccessible to anyone you have not authorized.
That's not very open, now is it? In fact it sounds like the author assumes that all 8 billion people in the world will all be running their own company, and they will all still be competing in a game of capitalism.
None of it is wrong exactly, but it feels like same enterprise-security machine finding the next anxiety surface than a "world is on fire right now" concern.
All of it always ends as a priesthood and a six-figure governance platform, rather than just taking practical steps to improve process.
For example, I could imagine a future AI telling me that it has modeled my behavior and built a very large differential equation that seems to perfectly fit my ideal pattern to maximally achieve my life goals and it looks like something Ramanujan came up with, and I'd tell it "that looks great, let's optimize my life based on that" while having no ability to even approach understanding something like that.
But the author's vision is also suspect, if you assume that the models will become much more intelligent:
1. Hypothetically, we can't give every human their own personal SkyNet to command. That would, uh, probably end very badly. If everyone gets an agent, those agents can't be too capable?
2. If you do somehow build a model that's much smarter than you, what do you contribute by managing it? How many people here have ever worked for a well-intentioned manager who couldn't understand the people they managed? So in this scenario, human management would be mostly displaced by agent management. Most companies could lay almost everyone off and let the agents manage each other. We only need humans to manage models now because the models are still pretty broken.
3. If we create models that can genuinely replace humans at almost any task, you won't be able to buy those on the API. At that point, the billionaires and the politicians wouldn't need human workers any more, because everything can be done better using their pet agents. Just have the robots build stuff for the billionaires directly. And if any of the former human peons get upset about being locked out of the economy to starve, then have the agents pilot the drones, too.
Basically, almost none of the people imagining a future of superhuman intelligences have actually though through how it would actually work in the real world. We're going to spend trillions of dollars and vast amounts of resources chasing the goal of making ordinary humans obsolete. Now, that goal might be unobtainable, I hope. But I'm deeply alarmed at how much we're spending pursuing it.
For a slightly less dramatic version - you can fire a human if they consistently do the wrong thing despite being told how to do it better.
Putting a big ball of matrix arithmetic on a Performance Improvement Plan makes no sense.
But what if companies that don’t track responsibility outcompete those who do?
In particular what if some perfect AI decision making ends up nailing decisions that maximize the expected reward for the company. And, well, if that comes at the cost of some unmanaged low-probability catastrophic risk, the company doesn’t care because all the decision makers are AI that don’t mind being shut off.