Better Models: Worse Tools(lucumr.pocoo.org) |
Better Models: Worse Tools(lucumr.pocoo.org) |
Edge has it in edge://flags no extension needed. Firefox & brave requires settings and extension like chrome. I haven't used opera or Vivaldi in ages to help with them, but they will have an option because customizability is a key part of their selling points
It's amazing anyone watched the last 2 decades of tech's enshitification and wants to hook their wagon to this shitshow.
> My strongest hypothesis is that this is not random deterioration but a training artifact. [...] Anthropic’s own client appears to expect and accept a fair amount of slop and repairs it, mostly silently
> If reinforcement learning happens in a harness like that, or a simulation of one, then slightly malformed tool calls can still complete the task and receive reward.
> Worse, the model may become very strongly adapted to the canonical Claude Code edit tool shape.
> Tool schemas are somewhere in the distribution and some shapes are close to what the model saw during post-training and some are far away.
Great article.
Interesting root cause hypothesis. Couldn't one simply strip the slop-handling from the RL env's harness to avoid this though?
I do agree on the walled garden being built here. Proprietary frontier models performing best in proprietary harnesses makes sense for Anthropic's interests.
I definitely think models may be trained to use particular popular harnesses or expect certain fields in the editing-tool or other tool schemas. Rather than trying to conform to (or force) one particular format, my approach instead is to design flexibly enough to handle a wide array of possible inputs and tool calls, but that also help the agent recover whenever its tool calls truly can't be salvaged and have to return etrors, and to auto-normalize results whenever reasonable to do so. It really does make a very dramatic difference (I wouldn't have bothered to launch if I thought it wasn't a meaningful advance) but anyway, just wanted to share my perspective given that I live and breathe this problem all day, every day.
I have gone back and forth between Claude and Cursor and it is clear Claude just throws the kitchen sink at problems to get an edge. I write MCP tools and I see these exact problems when the inputs and outputs aren't clearly defined, the LLM just guesses and retries.
Claude always gets the syntax wrong on my tool calls.
So I did a revolutionary thing and made the error output print helpful guidance on how to correctly call the tool.
The agent tries again and always gets it right. Total time “wasted”: 1-2 seconds. It happens every session, but it only happens once per context window. After that the agent holds on to the lesson.
To do this for your own tool calls, imagine what you’d do in the agent’s place - what info you’d need so you can correct your mistake. Assume the agent wants to achieve the goal so it’ll try again. These are probabilistic systems, so we need to give them an extra loop to get the deterministic bits right.
Almost no "solutions" in engineering/programming comes for free, one way or another, it's all a balancing act between different solutions with different tradeoffs. In this case, another request/response seems preferable to the other tradeoffs.
It honestly depends on the model. For my pi-brains extension for pi
https://github.com/gitsense/pi-brains
I've found after the first hook injection they get it, but there are occasions it can forget, but since everything is driven by hooks, you can inject as often as needed.
The issue with skills is, they are a one time thing, so you really can't use skills to correct haviorial issues.
Have a look at the TDD guard at https://codeleash.dev - the scripts/tdd_log.py arguments are pretty specific but it also has guidance in CLAUDE.md and lots of helpful error messages.
Not all; error messages are part of UX design, and the user error message should always give an error that indicates what the user can do to fix the problem.
If you cannot open a file for writing, don't just return "error: cannot open MyFile.txt", return "MyFile.txt: permission denied" (so user can request additional permissions from whoever), "MyFile.txt: no space left on device" (so user can free up some space), "Myfile.txt: file exists and is a directory" (So user can retry with a different name, or remove the directory, etc).
I think what is happening now is that, with so many of the agent-using pool of devs having never shipped to end-users before, they are surprised that their "program" (the tool) is being used wrong by the end-user (the LLM).
Those of us with battle-scars already expect the user to use it wrong and have learned that it's easier to tell the user how to fix the problem than to ask the user to read the manual/do it the correct way.
It obviously may end up in thousands of tokens burned through though (you can also fix that adding different levels of verbosity), but hopefully errors are not common.
If the agent uses the to incorrectly, validation fails.
If validation fails for ANY reason, print a message saying “here’s how to use it correctly”.
You don’t need to anticipate every misuse, just validate your inputs.
I use hooks to enforce TDD but the agent has to use bash tool calls to call a script before the hooks allow it to do its job.
tdd_log.py is a short python file with a #! at the top to make it executable, that (due to other hooks) the agent must run to declare its intent to make some tests red or green.
Check the info I posted in a sibling comment: https://news.ycombinator.com/item?id=48790589
The curl command is extremely popular so models seem to be really good at using it.
Also I like that curl uses a bash syntax and my platform requires JSON payloads; it makes the separation clear to the agent. I find it to be very reliable.
But:
"Now I’m somewhat worried about the track we’re on here. Alternative tool schemas might not just be unfamiliar. They might be implicitly punished by post-training that optimizes for one particular, forgiving tool ecology."
Only implicitly?
--
Many decades ago when I was working on research related to using MOOs as a learning environment, you would add "tool calls" into the stream of text that a MOO object might generate, so your rich client would e.g. show a picture, load a web page in a frame, move you on a map, trigger a change in an on-screen representation of an object.
Everyone who tried this in MUD/MUSH/MOO clients ran into more or less the same problems that LLM clients do: any attempt to shoehorn control sequences into in-band content was riddled with security risks, objects accidentally triggering the wrong interface etc.; you could never truly communicate out-of-band.
The more I read about how agentic harnesses work, the less embarrassed I feel about the code twenty-something-year-old me wrote in a MOO client.
Doesn't always work, for better performance you can kneel and start begging
The rationale: Anthropic's own harness (Claude Code) uses PascalCase tool names — Bash, Edit, Read, Write, Glob, Grep. Since the models are post-trained/aligned against that harness, those naming conventions are effectively baked into the model. Matching your harness's tool names to the same casing puts your inputs closer to the training distribution, which lines up with the more reliable tool use I've seen in evaling.
A related pattern that fits the same distribution: for long outputs, have the model reserve placeholders first and complete the work across multiple steps.
Reference: https://github.com/evotai/evot/commit/765151796c43965964a9da...
Pi is doing that.
I'd not discount the adversarial end of the spectrum.
Edit: found it, it’s called Grammar-Constrained Decoding (GCD)
EDIT: It's still quite fascinating seeing the kinds of things the models keep trying to do. It almost seems like when a human has slightly off with their nervous system. The conscious brain wants to do one thing, but for some reason the signals aren't getting to the hands correctly.
Is this still a thing? I thought Anthropic walked back the silent downgrades so now all the different domains downgrade non-silently.
There is still a downside, sometimes the model really wants to include an additional argument for whatever reason it reasoned towards, and it needs the error message to understand that the argument doesn't exist. Otherwise if the argument is manually removed from it's tool call, the model will think that it accidentally left out the argument and start retrying and might go into a loop.
- All models are terrible at generating line numbers for a proper diff, give up on them
- Some models (Owl-alpha) must have been post-trained on Codex transcripts, because they occasionally push its V4A patch format into any diff tool available
- Codex puts a lot of info in its system prompt about the desired patch style, making larger hunks instead of granular ones, etc
Only need ~650 tokens of system prompt for it to work. It’s pretty stellar.
You might want to run in some kind of sandbox to prevent the LLM from taking over the world, security is an isssue. But apart from that why not make the LLM write shell-programs instead of relying on JSON etc. ? Shell-scripting is the language for controlling the OS.
Once permissions are safe and secure and easy to set up and understand then using shell-programming as a general interface between agents and your PC/OS might be a good option.
Come to think of it I recently read that Microsoft is planning to produce an AI-oriented OS. Maybe agents with user-accounts is something they are aiming for, to solve the problem discussed in the article.
Things are not quite AGI yet; which is why people are now saying that intelligence is the harness + model, because the harness makes up for limitations in generalization.
That is not the case. There are some subtle differences between subscription and regular inference API, but not to the degree that behaviors change entirely. In Pi we're doing tests against both API and subscription API regularly to see how they behave.
Pro tip: Don't just print the return code, also print the call and it's arguments that failed, even without a stack trace.
Hooks can run code.
Hook code can be written in advance by the agent, runs in milliseconds, costs zero tokens, and gives the same result everytime.
Agents live at the boundary of codification; anything codifiable should be codified rather than run through an unpredictable machine. Hence, use hooks when you want determinism & predictability & certainty.
Examples: your stop hook could run tests against the code that’s just been written. Now, if your agent docs also tell your agent that the stop hook will run tests and there’s no need to run tests itself, then it’ll trigger a stop when it’s done instead of running tests itself. Just be sure to change the exit code to 2 and route the test failure output to stderr so Claude Code will show that output to the agent. Because the stop hook will fail over and over until tests pass, you just created a very simple guard that guarantees tests pass before you see the code - your agent can’t stop working without passing tests!