You need to rewrite your CLI for AI agents

You need to rewrite your CLI for AI agents(justin.poehnelt.com)

163 points by justinwp 74 days ago | 67 comments

dang 74 days ago |

Related ongoing thread:

Google Workspace CLI - https://news.ycombinator.com/item?id=47255881 - March 2026 (136 comments)

sheept 74 days ago |

This feels completely speculative: there's no measure of whether this approach is actually effective.

Personally, I'm skeptical:

- Having the agent look up the JSON schemas and skills to use the CLI still dumps a lot of tokens into its context.

- Designing for AI agents over humans doesn't seem very future proof. Much of the world is still designed for humans, so the developers of agents are incentivized to make agents increasingly tolerate human design.

- This design is novel and may be fairly unfamiliar in the LLM's training data, so I'd imagine the agent would spend more tokens figuring this CLI out compared to a more traditional, human-centered CLI.

gck1 74 days ago | |

Yeah, people seem to forget one of the L's in LLM stands for Language, and human language is likely the largest chunk in training data.

A cli that is well designed for humans is well designed for agents too. The only difference is that you shouldn't dump pages of content that can pollute context needlessly. But then again, you probably shouldn't be dumping pages of content for humans either.

Smaug123 74 days ago | | |

It's not obvious that human language is or should be the largest amount of training data. It's much easier to generate training data from computers than from humans, and having more training data is very valuable. In paticular, for example, one could imagine creating a vast number of debugging problems, with logs and associated command outputs, and training on them.

rkagerer 74 days ago | | |

I also feel like it's just a matter of time until someone cracks the nut of making agents better understand GUI's and more adept at using them.

Is there progress happening in that trajectory?

magospietato 74 days ago | |

Surely the skill for a cli tool is a couple of lines describing common usage, and a description of the help system?

sheept 74 days ago | | |

Sure, but the post itself brags,

> gws ships 100+ SKILL.md files

Which must altogether be hundreds of lines of YAML frontmatter polluting your context.

mellosouls 74 days ago |

John Carmack made this observation (cli-centred dev for agents) a year ago:

LLM assistants are going to be a good forcing function to make sure all app features are accessible from a textual interface as well as a gui. Yes, a strong enough AI can drive a gui, but it makes so much more sense to just make the gui a wrapper around a command line interface that an LLM can talk to directly.

https://x.com/ID_AA_Carmack/status/1874124927130886501

https://xcancel.com/ID_AA_Carmack/status/1874124927130886501

Andrej Karpathy reiterated it a couple of weeks ago:

CLIs are super exciting precisely because they are a "legacy" technology, which means AI agents can natively and easily use them, combine them, interact with them via the entire terminal toolkit.

https://x.com/karpathy/status/2026360908398862478

https://xcancel.com/karpathy/status/2026360908398862478

lidn12 74 days ago | |

Thanks for sharing these contents. They are very interesting. I found "making all app features accessible from a textual interface..." actually quite challenging in cerntain domains such as graphics related editing tools. Though many editing functions can be exposed as CLI properly, but the content being edited is very hard to be converted into texts without losing its geometric meaning. Maybe this is where we truly need the multimodal models or where training on specialized data is needed.

Terretta 71 days ago | | |

> the content being edited is very hard to be converted into texts

For decades now, pro design print shops have required text files describing the design to print from.

And as every Danish pelican cyclist knows, graphics are their most scalable as text vectors.

Inkscape does fine with these.

utopiah 74 days ago |

That's how artificial this "intelligence" is, when LLMs can't even use text based tools full of txt based documentation formatted coherently without those very tools being adapted.

bonoboTP 74 days ago | |

It looks like an AI generated fluff article without any evidence. People also did this for image generators as if you needed these arcane templates to prompt them, but actually the latest models are great at figuring out what you want from messy human input. Similarly LLMs can use regular CLI just fine. But how do you write a hype FOMO article about the fact that actually you don't need to do anything...

jsunderland323 74 days ago |

I'm working on a CLI now.

The pattern I used was this:

1) made a docs command that printed out the path of the available docs

$ my-cli docs

- README.md

- DOC1.md

- dir2/DOC2.md

2) added a --path flag to print out a specific doc (tried to keep each doc less than 400 lines).

$ my-cli docs --path dir2/DOC2.md

# Contents of DOC2.md

3) added embeddings so I could do semantic search

$ my-cli search "how do I install x?"

[1] DOC1.md

"You can install x by ..."

[2] dir2/DOC2.md

"after you install..."

You then just need a simple skill to tell the agent about the docs and search command.

I actually love this as a pattern, it works really well. I got it to work with i18n too.

danw1979 74 days ago | |

I really like this - especially the embedded search. What do the embeddings and model cost you in terms of binary size ?

jsunderland323 74 days ago | | |

Umm it's not as bad I thought it would be. ~16mb per locale. ~28k words in the english docs.

Not schilling, just easier to show you the repo since it's open source. https://github.com/coast-guard/coasts

jeppeb 74 days ago |

The article states that agents work better with JSON than documented flags - that seems counterintuitive . How is this assumption validated?

justinwp 74 days ago | |

Try building a CLI with a complex JSON as flags approach. :)

peddling-brink 74 days ago | | |

I just did the opposite and am seeing better results.

Claude was struggling to use the ‘gh’ command to reliably read and respond to code review line level comments because it had to use the api. I had it write a few simple command line tools and a skill for invoking it, instantly better results.

YMMV

smy20011 74 days ago |

Are we reinventing RPC again? Calling CLI program with JSON sounds like RPC call. The schema feels likes something LSP can provided for such function.

Maybe asking agent to write/execute code that wraps CLI is a better solution.

CamperBob2 74 days ago | |

Confused? You won't be, after this week's episode of https://en.wikipedia.org/wiki/SOAP.

Everything old is new again...

tayo42 74 days ago | |

Doesn't powershell have structured input and output?

_heimdall 74 days ago |

I strongly disagree here. Yes, build CLIs. No, don't target then at agents.

Build for humans, including good man pages or `--help` docs as needed.

If LLMs are worth the name AI they will understand how to discover and use Unix-style commands. In my experience, this is exactly the case and I need only say "I use tool X for use case) Y."

danirod 74 days ago |

With all due respect, but if humans can figure out how new unseen programs work by using -h and seeing what options exist and what they do, I am sure robots can figure it out too, or else they weren’t that intelligent to begin with.

theshrike79 74 days ago | |

The difference is that humans will remember the options listed by -h after a few times of using the tool.

AIs don't. If they don't reach for the --help switch every time they'll attempt the statistical average, which may or may not work.

For super-common or popular tools like `gh` the usage is already in the training data though.

impure 74 days ago |

It’s always amusing when the all powerful agents need our help to work properly.

Them: Agents will write code 10x faster than a human.

Also them: You will have to dumb down your CLI for them to function properly.

stratifyintel 74 days ago |

One thing I keep wondering about with agents is what happens once they start running real workflows.

Right now a lot of the focus is on getting them to execute tasks reliably. But the harder problem might be reconstructing what actually happened later, why a decision was made, what context the agent had, and how you audit or override it if something goes wrong.

Feels like autonomy is moving faster than the accountability layer around it.

justinwp 74 days ago |

Title is a little clickbait, but not really! :)

---

Human DX optimizes for discoverability and forgiveness. Agent DX optimizes for predictability and defense-in-depth. These are different enough that retrofitting a human-first CLI for agents is a losing bet.

bingemaker 74 days ago |

If AI agents need CLIs, then whats stopping them from using APIs directly. I see CLIs as good wrappers over APIs, and nothing more. What more will CLIs provide which `curl -X POST` can't/won't provide?

zihotki 74 days ago | |

Context limits and context poison/rot/whatnot stops them. CLI's are a great way to make a focused context. But any other trick to filter out noise would also work.

Romamos 69 days ago |

I have to add my two cents.

1. Deterministic & Non-Blocking Execution(--yes)

2. Not just Structured Outputs but strictly validated Inputs also("count should be a valid integer")

3. Actionable Structured Errors("")

4. Semantic Exit Codes(0 - ok, 1 - input error, 2 - critical)

5. Granular Verbosity Control(-v, -vv, -vvv)

6. Composable Structured Pipes

jauntywundrkind 74 days ago |

And for more persistent services, worth considering using varlink, for your agents sake and just if you need two cli thinks to chat. https://varlink.org/

The systemd universe is moving this way from dbus, and there doesn't seem to be a ton of protest against giving up dbus for json over unix sockets. There's really not that many protocols that are super pleasant for conversing with across sockets.

lynx97 74 days ago |

I love how AI gave the command-line and TUI interfaces a kind of Second Renaissance. It is not just AI that loves CLIs. It is especially blind people like me, who still use a lot of text-mode tools for their implicit accessibility. I gave codex a whirl recently, and hey! No accessibility problems at all. Just works. A few years back, that would have been released as a GUI-only program and would have locked me out completely[1]. A blessing that text oriented interaction is becoming important again!!!

1: Strictly speaking, there are ways to access some GUI programs on Linux with a screen reader. However, frankly, most are not really a joy to use. The speed of interaction I get from a TUI is simply unmatched. Whenever I work with a true GUI, no matter if Windows, Mac or Linux, it feels like I am trying to run away from a monster in a dream. I try to run, but all I manage to do is wobble about...

utopiah 74 days ago | |

A second renaissance? The entire time internet had been running on CLI. All modern services relying om containers rely on images based on CLI. There is no renaissance needed because it never stopped.

lynx97 74 days ago | | |

Sure, server-side. What I am talking about is user-facing programs. That is why I mentioned codex as an example.

mahendra0203 66 days ago |

Honestly? Don't think you don't need to rewrite your CLI.

The whole "raw JSON > flags" argument sounds clean in a blog but falls apart in practice. Agents are actually good at parsing messy human output. stderr, exit codes, even tables.

I run AI agents against CLIs every day. The failures are almost never, "the outpt wasn't structured enough." They're context window overflows, permission issues, and the agent confidently running the wrong command over and o ver.

The real insight here shouldn't be "rewrite for agents" — it's "make your CLI not actively hostile." --output json and non-zero exit codes on failure. That's should be it.

Maybe we are optimising the wrong layer..

resiros 74 days ago |

I don't follow the need to write CLIs for the agent. Why not use simply the API and document it well? The token difference between using an API and CLI is not that much, and models are trained to use REST APIs and understand their patterns, compared to your random CLI.

devmor 74 days ago |

This is a very silly premise to begin with.

If AI agents are so underdeveloped and useless that they can’t parse out CLI flags, then the answer is not to rewrite the CLI.

You either give the agents an API layer or you don’t use them because they’re not mature enough for the problem space.

rsanheim 74 days ago |

No. Nope. Agents do just fine with all sorts of CLIs. Old standards, new custom stuff, whatever.

The CLIs I’ve seen agents struggle with are those that wrap an enormous, unwieldy, poorly designed API under one namespace. All of Google Workspace apis, for example.

danw1979 74 days ago | |

Which is perhaps why the author was focussing on optimising LLM usage of such unwieldy APIs without blowing out the context ?

I don’t disagree with your point about agent abilities with older, concise, well-represented-in-training-data tools though.

computerfriend 74 days ago |

> Humans rarely typo a traversal.

I don't think this is true?

MattGaiser 74 days ago | |

The typos are a bit different, but that’s one reason I hate the command line as a human.

You want me to hand type a file name? I’ll flip a letter or skip one!

bewuethr 74 days ago | | |

I like to let tab completion write my filenames for me whenever possible.

esseph 74 days ago | | |

I tab complete and fish shell also gives a nice little arrow selectable menu if options

pickleglitch 74 days ago |

Counter point: No I don't.

inkdust2021 74 days ago |

There are so many diffierent CLI, why not use them or basic on them to rewrite?

lejalv 74 days ago |

> The real question: what does it actually look like to build for this?

What was the not-so-real question? Or the surreal question?

I know it's becoming tiresome complaining of slop in HN. But folks! Put a bit of care in your writing! It is starting to look as if people had one more agent skill "write blogpost", with predictable results, as we are not a Python interpreter putting up with meh-to-disgusting code but actual humans with real lives and a sense of taste in communication

vasco 74 days ago | |

The content is good, who cares. What gets more tiresome is complaints like this. The post was upvoted so people like it.

redanddead 74 days ago | |

blogslop

Dansvidania 74 days ago |

In the meantime I need to go through captcha to access the blog…

alexruf 73 days ago |

I absolutely disagree. Tools should always be built for humans, not for machines! Believe it or not, I'm still one of those people who is actually capable of operating their own computer and doing meaningful work without any AI assistant (and I am proud of it). If that possibility or ability is taken away from me because the applications on my computer can only be operated through an AI — then I'm out and start a carer in woodworking or something.

climike 74 days ago |

cliwatch.com, creating some benchmarks, reach out if you are interested :)

948382828528 74 days ago |

I do not.

antisol 74 days ago |

Why are you calling it "AI" if it can't even parse a goddamn man page?

theshrike79 74 days ago | |

It can, but it pollutes the context with useless information costing you money.

lloydatkinson 74 days ago |

No I don’t thanks.

donpark 74 days ago |

I suspect this CLI madness is just a phase, but if the trend continues, I trust AI agents to handle the necessary rewrites.

bayindirh 74 days ago |

Haha, no.

I write my tools for humans, without help or use of AI. If the AI agent wants to use my tool so bad, they need to rise to that level. I'll not crouch on my knees to meet it.

If I ever write a tool for AI interaction, I'd give it a well-defined API, to make it even easier for the agent.

ZeroGravitas 74 days ago |

Some of this seems a bit overhyped.

I like CLI tools with json output that can be piped through jq. I've seen llms do that with existing tools.

The human needs and llm needs seem to overlap, especially if the human is using scripts and piping between tools.

The number of times it implies you didn't need to validate "human" input until llms arrived is scary too.

I'm also surprised to hear them say llms shouldn't Google, as that seems an area that Google themselves could optimize their search service for their llms and get an advantage.

Finally, I wonder if just using older, smaller llms is a valid fuzzing approach for clis (or anything else llms might be controlling). Or do you need a high powered llm trained to generate adversarial input?

danw1979 74 days ago | |

> The number of times it implies you didn't need to validate "human" input until llms arrived is scary too.

I took away a completely different message: humans and LLMs make different mistakes that require different validation.