Quoting moozilla's comment 11 months ago (https://news.ycombinator.com/item?id=44741894):
---
If anyone is curious on the context: https://x.com/thdxr/status/1933561254481666466 https://x.com/meowgorithm/status/1933593074820891062 https://www.youtube.com/watch?v=qCJBbVJ_wP0
Gemini summary of the above:
- Kujtim Hoxha creates a project named TermAI using open-source libraries from the company Charm.
- Two other developers, Dax (a well-known internet personality and developer) and Adam (a developer and co-founder of Chef, known for his work on open-source and developer tools), join the project.
- They rebrand it to OpenCode, with Dax buying the domain and both heavily promoting it and improving the UI/UX.
- The project rapidly gains popularity and GitHub stars, largely due to Dax and Adam's influence and contributions.
- Charm, the company behind the original libraries, offers Kujtim a full-time role to continue working on the project, effectively acqui-hiring him.
- Kujtim accepts the offer. As the original owner of the GitHub repository, he moves the project and its stars to Charm's organization. Dax and Adam object, not wanting the community project to be owned by a VC-backed company.
- Allegations surface that Charm rewrote git history to remove Dax's commits, banned Adam from the repo, and deleted comments that were critical of the move.
- Dax and Adam, who own the opencode.ai domain and claim ownership of the brand they created, fork the original repo and launch their own version under the OpenCode name.
- For a time, two competing projects named OpenCode exist, causing significant community confusion.
- Following the public backlash, Charm eventually renames its version to Crush, ceding the OpenCode name to the project now maintained by Dax and Adam.
Nobody is reading the issues; nobody is accepting PRs.
>You avoid the uncanny valley where the model appears to be intelligent before doing something stupid; the stupidity is self-evident and this helps calibrate your interactions.
This seems to be capturing a feeling I am finding messing with local LLMs: popular software around this seems to be slop coded by no talent hacks. It's a common experience to see a python process pegged at 100% CPU with a huge amount of memory allocated, seemingly doing something that could be a simple shell script. That is before stuff reaches the LLM.
If that runs on somebody else's machine, you don't notice. When you see it in top it's frustrating.
But are there are any reasons a pragmatic and informed developer would use OpenCode vs Claude/Codex as a harness today?
I'm not seeing anything competitive in terms of cost/quality/trust?
That said, I firmly believe that if AI is to survive, the future HAS TO BE local or near-local. Having said that, statements such as
> Docker causes security holes:
> It creates a god-service that runs as root.
> It deliberately punches a hole in ufw firewalls.
Sorry, none of those are correct IF you know what you are doing. Though I will admit, seeing people that know what they are doing is increasingly uncommon.
Also there is nothing wrong with developing inside containers. If anything, that is arguably one of the biggest selling points for containers - environment(s) you can crash infinitely at no cost.
I still dislike opencode for a bunch of reasons - the assumption that llms are immune to screw ups, being one. As for the default behavior - using cloud by default - I didn't know that(I do not use any AI for direct coding tasks) and if that is the case, yeah, this is bad. Undeniably a horrible decision.
Some of my own frustrations with bad Docker deployments leaked through. I did snip some of them out way back when I posted this, but maybe not enough.
Essentially, what you want is that the people who produce harnesses like OpenCode to also implement proper sandboxing and security. Why... why do you believe they'll do anything remotely satisfactory?
It's been empirically demonstrated, again and again, that at least 85% of software developers (and this is a very generous estimation) can not be trusted to bother about the security, so leaving the implementation of the security mechanisms and policies up to them simply doesn't work in practice. It just doesn't.
https://github.com/1jehuang/jcode https://github.com/gi-dellav/zerostack
In a years time the discussions on using CC, Opencode, Pidev etc. will be redundant as we will be building our own tooling with whatever programming language and tooling you want.
Some will repurpose a popular opensource tool or harness, others will build it from scratch in an hour or so.
I mean, there's a perfectly good solution for many of the security issues mentioned in the article... But it is dismissed because the author just doesn't "want to"?
By the way, LXC is probably better suited for reproducible build/development environments, over Docker.
The remote-preferred attitude caught me once. Also, its server APIs don't match its docs, if you want to talk to it programatically.
The following config works for me to keep it on a local model:
{ "$schema": "https://opencode.ai/config.json", "provider": { "local-llama": { "npm": "@ai-sdk/openai-compatible", "name": "Local llama.cpp Server", "options": { "baseURL": "http://localhost:8080/v1" }, "models": { "Chefs Choice": { "name": "ANY" } } }, ...
This allows me to put any model I want on port 8080.
I understand why the mods do it, but I don’t appreciate it. It’s your space, but it’s not your article.
May as well write a post saying don't use LLM's for any SWE work.
>Conclusion Stop using OpenCode.
>Post-script: Local LLMs This is worth its own post – I have multiple attempts in my blog drafts – but it needs to be addressed briefly here. My opinion on local LLMs like Qwen3.6-27B is they are corrosive to the stability and conceptual fidelity of your codebase in the same way as frontier models, with the following three differences:
>You avoid the uncanny valley where the model appears to be intelligent before doing something stupid; the stupidity is self-evident and this helps calibrate your interactions.
>The weight count is too low to reproduce the training set verbatim, which nudges the calculus on whether the output should be considered tainted. This is distinct from larger models which can reproduce inputs verbatim, but are trained to refuse to.
>You avoid supporting or relying upon cloud providers.
>I’ve had useful results from input-oriented tasks like: “I think there is a bug in code x with symptoms y, my guess on the mechanism is z. Read all relevant code, come back with a call chain and code citations.” Framing it as a search problem reins in the clanker’s propensity to make shit up.
>Using LLMs for code generation feels like a dead end. However thoroughly you think you understand your architecture, your planning is constantly undone by shortcuts like “what if I just move this mutable state into the middle of the design so everyone can share it?” This is hostile to your ability to understand your code, beyond the fact that you didn’t write it.
>Drawing answers directly from knowledge in model weights leads to hallucination even for multi-trillion-parameter models, so why bother making them that big? If people were realistic about limitations then we wouldn’t be building new power stations for datacenters, and they wouldn’t be rammed into every product.
>The entire software ecosystem around LLMs is completely rotten, and if they do ever become “just a tool” then some actual systems engineering needs to be done around them to turn them into tools instead of security black holes. That work will have to be done by humans.
> May as well write a post saying don't use LLM's for any SWE work.
Alternatives exist
Um. Which then gets immediately cached again.
I prefer this behavior. I prefer my changes incorporated immediately, rather than wait for the next session. Call me crazy.
(Author is unhinged and this story is upvoted because #AI-HATE.)
I might accept if my newly saved information was overlooked, but it's hard for me to fathom how I'd demand "I want to save this information in AGENTS.md now, but I don't want you to look at it until the next time I launch you..." and call any behavior other than that annoying or alarming.
(And also your point about immediately being re-cached a good one)
Plugin opencode-slim-system can be used to override the system prompt and the tool description for any tool (including bash). It helps a lot to fix their nonsense.
magic-context plugin fixes the compaction issues.
Run the entire process in landstrip for sandboxing to alleviate any bash concerns.
Many users did things right the first couple of days and then turned the security off after seeing it 'work fine' on its own.
Was a cool thing to use for a couple of weeks in v1, you have to be quite static to keep using it at v68 of self vibecoding while the llm providers like openai already developed their in house alternative.
I mean this is a problem with many coworkers too, so you deal with it in the same way: limit what they can do to creating pull requests.
Author is unhinged.
Ignoring the appeal to authority inherent to this kind of statement, it's ignoring another implication. Large companies can afford to maintain a fork. Using the latest release of OpenCode day to day is a different proposition than serving as some corporate developmental branch "base", as shorthand for all sorts of changes/usage.
...
[chadjak.png] Your terms are acceptable.
What's the alternative there? Gemini used to have free tiers. Qwen used to. AMP used to. Ollama cloud used to. Codebuff used to... None of them have those programs anymore
The reason I use it is purely financial. I do not have an employer and I'm writing free software
Edit: pi.dev doesn't have free inference endpoints. That's the constraint.
I've written my own. I know they exist.
I don't offer free inference on large models however
But the knife cuts both ways. As you speak, so will your heart be. I think modern psychology supports this. Actions drive attitude, and attitude drives actions. If you assert your will in either place, the other is disrupted and will adjust.
You can literally change yourself into whoever you want to be. But a passive participant in this process will simply show the world what kind of person they are without ever realizing it.
There have been some attempts to approximately stitch KV blocks without full re-evaluation (see: CacheBlend on arxiv), but the results aren't promising.
You mentioned HiCache: as far as I can tell this is a prefix tree cache that efficiently shares prefixes across related histories. If you change the prefix, everything after the change still misses the cache and needs to be prefilled again.
Again if I can get this somewhere else then I'm game.
I'll go check that but this would be fairly shocking
nemotron-3-ultra-free big-pickle north-mini-code-free mimo-v2.5-free deepseek-v4-flash-free hy3-free
Alright.
# Prompt Cache Misses
> It globs your filesystem and re-reads AGENTS.md (injected in turn-0 system prompt) on every SSE turn. If you put a quick note in AGENTS.md to be read in the next session, you immediately force a full re-evaluation.
> Personal favourite: it puts the current date in the turn-0 system prompt and re-evaluates every SSE turn. If you’re using OpenCode at midnight you get a full prompt cache miss.
Okay, I can live with those.
# Compaction
> Want to sit for 10 minutes while the LLM server prefills the entire session with a new prompt prefixed to it, just to turn it into 5 bullet points that go at the top of a new session? Me neither. I get what they are going for, but I’ve not seen it work well. Neither compaction nor pruning is implemented well, and they interact poorly.
Is this an OpenCode specific issue? I've seen the same with Codex and Claude
# System Prompts
> The default system prompt is opinionated (fine) but it has shit opinions (not fine). It took me a while to figure out why my agent kept saying “Use ABSOLUTELY NO COMMENTS” when dispatching subagents.
Okay, so change it? Any LLM is opinionated, this system prompt enforces consistency across different models which seems reasonable.
I liked Opencode a lot previously but there's no doubt it's not well-written software. Pi has fully replaced it for me and there are quite a few newer options that learned from Opencode and exercise more minimalism where it matters
It hardly matters that it's vibe coded and ugly if it works well... and it does (for me).
Saving a few CPU cycles and reducing RAM utilization on my local system isn't going to meaningfully improve anything in and of itself; the wall time is all bounded by the model or by the code it's invoking (i.e. compiling or running the test suite).
Just wanted to add a few things:
- We actually don't do tool-call pruning any more. But compaction is a necessary evil for the time being. There's only so much context window, and if you wish to keep working on the same task for a long time, the model needs to summarize its current progress so it can continue without hitting its token ceiling.
- With V2 (in beta atm), we have a new system for handling changing system instructions (AGENTS.md files, available skills, etc.) that avoids cache misses at all costs, while still giving the robot the newest information. I recorded a little video about it here:
I updated one of my opencode containers today and had to downgrade it back because:
1. Worktree/workspace support is gone altogether. No way for agent to work in parallel on different things safely except prompting it directly to create a worktree.
2. Seeing multiple sessions properly now requires opening tabs for those?
3. In general, switching between several projects/sessions/whatever stopped working, and command palette is hotkey-only.
Issues about the new design just sit in an endless unsolved pile on GitHub which seems hopeless. And an agressive stale bot is not helping with that.
I came to OpenCode because it seemed hackable and simple, and at this point it seems like a "move fast break a lot" project that you got to live with.
(rant over, sorry if this is addressed to the wrong person, just needed to vent a bit)
Those are labeled as "annoying things"
Read also the "alarming things"
There is another section titled "Alarming Things", with a subsection titled "It’s Fucking Full of RCEs". (Those are RCEs other than those which simply derive from the issues pointed out in previous subsections).
So that's why OpenCode likes to randomly remove my comments.
I have the following in my instructions, but I often need to remind agents of it because they follow the shitty system prompt instructions:
"ALWAYS include MANY inline code comments describing what blocks of code are supposed to be doing. Inline comments serve as inline specification, a parity check between the code and the specification, and are a means to _communicate_ with all future programmers, including yourself. Write Once, Read Many. The code needs to talk to whomever is looking at it in natural language."
> It globs your filesystem and re-reads AGENTS.md (injected in turn-0 system prompt) on every SSE turn. If you put a quick note in AGENTS.md to be read in the next session, you immediately force a full re-evaluation.
> Personal favourite: it puts the current date in the turn-0 system prompt and re-evaluates every SSE turn. If you’re using OpenCode at midnight you get a full prompt cache miss.
I used opencode with deepseek v4 via openrouter during the weekend. Openrouter claims a cache hit rate of about 95%. Maybe it can be improved, but it doesn't seem broken.
I agree that it is bloated, and uses a lot of memory and cpu. But I don't remember seeing any bug during its usage, or at least not big enough to notice
Comments can always be stripped reliably later.
I think that is reasonable and intuitive.
That was the first complaint I came across in the article and it made me question the level of indignation of the whole thing.
> If you don’t know what OpenCode is, imagine a boot stamping on a human face forever. The boot is made of TypeScript and the face is everything we have learned about security and systems software since the invention of the electronic computer in the 1940s.
Regardless if that's a good or bad take, I'd like titles on HN to reflect what the author wanted to say and not editorialized to what HN readers think the article should be saying.
Sad but not surprised to hear about all these other issues. I like opencode, it does plenty right but plenty wrong.
> Okay, so change it? Any LLM is opinionated, this system prompt enforces consistency across different models which seems reasonable.
OpenCode embeds scatters bad coding policy all over all of its inbuilt prompts, including tool instructions. The only way I've found to fix them is to edit the source code. And, yes, they're also huge and repetitive.
A framework's native prompts should include only information about how to interact with the framework itself.
https://artificialanalysis.ai/agents/coding-agents?coding-ag...
It's honestly embarrassing to call this open source.
Pi on the other hand, whilst they have their own unique issues with PRs, if you came to them with a genuine problem, or fix, they seem to accept it.
Now I point back to those issues you pointed out, all have PRs... Unmerged. Same with the infinite depth subagent calling.
OpenCode has an MIT license, which allows you to copy/fork the code and modify it how you see fit.
Please review the open source initiative's definition of open source: https://opensource.org/osd
1. Most obviously & importantly this is a complaint without a straightforward suggested alternative. A sibling commenter mentions suggesting fixes to Opencode would be more productive: I don't necessarily agree since many of these issues are fundamental & would likely require an almost ground up rethink & rewrite, but the issue is that the article contains no constructive proposal at all: it may as well be titled "Stop Using LLMs"
2. None of the major issues listed are unique to OpenCode. At least the full list within the "Alarming Things" seems in my mind to apply to Claude CLI, & I would guess most other agents from frontier model providers.
Granted it's worthwhile documenting these issues as a plea for someone to build better tooling from the ground up, so the article is far from worthless - on the contrary I've bookmarked it & will be sharing & referencing it widely & often. But the title & focus is just very odd & seems misguided, especially when the contents of the piece is otherwise so good.
I nominate this for a Bulwer-Lytton prize in the Tortured Metaphor category.
George Orwell, 1984
> My conclusion is that OpenCode is clown-car turboslop with a security posture of “let me bend over for you daddy”. Everyone using it should stop using it.
I do not want to keep reading. There are regular people who wrote this software. When did it become normal to talk about open source like this? How would you feel if someone wrote this about some software that you wrote? Reading this article made me feel extremely gross
Because everything matches and explains oddities I've seen and forgiven. So despite the hyperbole and the places where the authors' general sentiment are not mine, he's basically right, and I need to find a different harness.
If the post resonates for you too and you've found something better, happy for recommendations. pi seems to get the most mentions here, anyone feel it (or another option) is specifically better on these architectural aspects?
These are all just some minor annoyances, some disagreements, and most importantly, a fundamental misunderstanding of the point of command filtering. It is not for security, it’s for steering.
Anyway, it didn’t sound like the author tried building anything with OpenCode, and if they did, they said nothing about the most important part: how well did it do?
I feel the same way. It's the right amount of 'gets out of my way' and 'won't nuke my machine'.
Especially the ease of using Plan mode and quickly getting stuff done.
OpenCode has also seemed to have disappeared from https://openrouter.ai/apps/category/coding
I've been working on my own AI harness/chat in C++ Qt6[1] and it's lighter than anything else even with lots of features like sub-agents, code diff, terminal emulator, simple editor, markdown preview, translucent background, custom themes, permissions, MCP, git integration(commits, stage, diffs), dock system to move elements and tabs for projects; but it's not released yet as I want to polish it a bit, remove a few bugs I found and simplify the UI.
Including the full datetime would be irresponsible, but that's not what OpenCode does.
A simple solution here would be to evaluate the date once per session, or once each time the `opencode` binary is launched (to avoid old, long-running sessions getting stuck in the past).
some things mention are outdated - particularly the tool call pruning feature. this has been disabled by default for a while specifically because of complaints like this
when we looked through our data it's not so clear cut that it's net negative https://x.com/thdxr/status/2048268697790300343
also i'm pretty excited about our new system prompt approach - can define each component in a way that avoids busting the cache when they change. eg that midnight issue: https://x.com/thdxr/status/2070721003924103651
providers (anthropic is the first) are supporting this as a native concept
I think the biggest concern from the article would be the open http server, and things like the fetched arbitrary command execution here: https://github.com/anomalyco/opencode/issues/22191
Was that issue addressed? The comment where it was marked as closed suggested it "needed more testing" but didn't have any linked PRs.
Gemini translates the closing comment as "We are closing this ticket after a quick look. The login issue is either caused by a wrong setting/external provider, or it's a deeper problem that requires someone to dig into the backend Go server code directly."
GAH!!! So that's why its been deleting comments. That is annoying as hell.
FWIW though, I think this applies to other harnesses also (not the annoying bits, but the security risks). I was thinking this morning about how much of a supply-chain vulnerability these tools are. Like, basically, they have a ton of data, they're updated practically daily, and given their vibe coded nature I really doubt anyone is auditing the 10,000 NPM dependencies they drag in. It really will just take one left-pad incident for this to be an absolute disaster.
Not if you only use an allow list. E.g. only allow things matching a particular prefix to run. Also, isn't this what every agent does? And by every I mean Codex because I haven't used the others.
Nevertheless I'm curious what people might suggest to use instead with local models.
This sounds identical to what OpenCode does (&, contrary to their own marketing, does not seem sensible at all).
The fact that this mechanism is so primitive across all agentic clis is frankly a serious indictment of the competence & experience behind LLM-focused development teams.
This is a security issue that has in the past been solved via explicit file-path-based allowlisting, & more recently - due to file-path matching being deemed insufficiently secure - apps have been switching to cryptographically signed binary matching as a more modern alternative to the less secure path matching. In that context, the fact these recently-written "modern" agentic clis are using a method that's more basic & even less secure than the "old insecure" method most apps use is pretty shocking.
My type of writing.
I may be missing some aspect of where the local llm’s are losing, but I’m content with draining my local tokens and treating the frontier models like a less junior SWE.
Instead what I read was typical issues about Agentic CLI's in general dubbed as `alarming`. I would have been happy if the title was just `Annoying` which a lot of `OpenCode` is. But adding the `alarming` made it what it became, something that attracts enough eyeballs to get to the front page of HN.
This article (combined with another article previously comparing CC and OC's system prompt, to which someone replied pi.dev sends even less) has made me want to switch agents, so I'm likely going to try out pi.dev and aim to migrate.
> Using LLMs for code generation feels like a dead end. ..., This is hostile to your ability to understand your code, beyond the fact that you didn’t write it.
What you get from AI is not worth of what you have invested for it in long period.
> Drawing answers directly from knowledge in model weights leads to hallucination even for multi-trillion-parameter models, so why bother making them that big?
Current AI is like the film company producing TV series or movies. "If similar stories do not exist or details are missing, screenwriters use imagination to fill in the gaps (remember hallucination? It's just a makeup.)", refer to the article https://devcyc.life/current-ai-is-like-the-film-company-prod...
> If you don’t know what OpenCode is, imagine a boot stamping on a human face forever.
> If you don’t know what AI is, imagine a boot stamping on a human face forever. The boot is made of shit and the face is everything we have learned about security and systems software since the invention of the electronic computer in the 1940s.
I have vscode running in flatpak with directory permissions handled by flatseal. Vscode only has access to my dev folders and nothing else. Even the git bundled by vscode cannot call git push because of this (vscode doesn't have permission to read ~/.ssh!).
It's an easy sandbox that's provided free of charge courtesy of bubblewrap/flatpak.
As someone who uses opencode regularly, the quip about it asking for permission to read logs in /tmp after already writing to the directory is pretty funny.
Useless, if you are dealing with privacy compliance and tons of legal requirements.
Making the authors stance on LLMs clear.
All of the major search engines have understood that as a stand-in for "Go the programming language" for many years.
It has overridden a 4000 year old strategy board game played by millions (mostly Asians).
It's written in Rust. It seems to perform well. It also seems (from my poking around in it) on the whole well and deliberately written. It doesn't consume memory like a mf'er like Claude Code. It doesn't sit and chew background CPU like I've seen opencode.
I suspect it will accumulate more specific-to-OpenAI-isms over time though.
Workspace support is being worked on, it was half-implemented before, so we are taking a proper pass at it now.
For switching between sessions, can you tell me about your workflow from before? Because a tab can hold a session from any project. Are you saying that opening up a tab is too heavy of an action?
GitHub issues though has been a problem from us, we deal with a ton of noise/spam.
I generally view the kind of code in projects like OpenCode to be the same as binary machine code, and the harness that works on it is the compiler. The source code is the prompts (I guess), except we live in a weird state now we use the existing machine code + more prompts to make even more machine code.
Not that anyone really cares here, but the gp quote isn't even really fundamentally a metaphor either way, its an exhortation to "imagine". The Orwell isn't even a metaphor!
Like many popular Ai projects, they get 1000s of slop PRs. They are also working on a v2, so fixes to old code are much less likely to land now.
If you need to, just prefix user messages with the current datetime.
Why is this even a complaint?
His complaint that "echo git | bash" still works is particularly nutty.
This works for the `bash` example too. Spawning a new shell and running arbitrary commands is not an escalation because the child shell inherits the parent's restrictions.
For example, I recently ran opencode via sandbox-exec & naively granted the sandbox access to the aws cli. First attempt to exec it failed because I hadn't granted the sandbox access to the credential file in ~/.aws. This took 2 seconds to resolve but effectively highlights how utterly useless the tree-sitter string-parsing they're doing is. It's so concerning because implementing tree-sitter for that is non-trivial to the point that any competent dev should surely realise quickly while implementing it that this avenue is utterly useless. Yet every high-profile agent cli has taken this approach.
In the old days of comp.lang.lisp, there were certain individuals who would really get off on denigrating people who wrote code that was not up to their ivory tower level. Some people left because of it. Others took it as a badge of honor falsely believing that the dressing down was necessary for them to improve their coding skills.
Example of an old discussion on HN about it:
> People familiar with OpenCode internals (if you are on the OpenCode dev team I assume this doesn’t include you) might have objected to my python3 example above.
It's not normal today, but it's also not new, that kind of language has been around for a long time.
Why does something being open source absolve it of any sins?
> a lot of apps run 100% of traffic through openrouter while for us it's a small subset
> but companies we were trying to work with used openrouter as a source of truth on popularity so they thought products that were much smaller were bigger than us
That's not because OpenCode is bad, but because Pi is good. I can say the same about Claude vs pi.
I would love to learn how LSP helps you. I did various basic evals with LSP and I did not find it to improve the agent performance at all. Maybe on newer models this gets better because they are now RLing with LSP in the context, but at least in the past having a commit hook that runs lints and typecheck at the end, is more token efficient than having every edit be injected extra LSP results. You are almost guaranteed to be wasting tokens since partial edits are unlikely going to result in type checks passing.
(Disclaimer: I work on Pi)
Having come from Claude code, I was a bit shocked that both seem to allow edits by default with no confirmation dialog.
One IIRC allows you to enable it in config and the other needs a plugin.
There are confirmation dialogs in OpenCode. You can configure them using the permission config, same as Claude Code. Having said that, as a general rule, none of these harnesses are safe to run on your local file system.
https://github.com/anomalyco/opencode/releases?page=21#relea...
Personally, I run pi-agent in a custom sandbox based on bwrap, with an internet proxy. This mostly limits the blast radius to one source tree and one git checkout. And I don't give it push/pull permission. Local models are fun in a "closely supervised but slightly clueless minion" sort of way.
But all of these agents have or had "dumb" security issues:
https://github.com/anthropics/claude-code/security/advisorie...
You probably know that since you run pi inside https://github.com/containers/bubblewrap.
yes it's bad if the permission system is broken, but serious users have not trusted this stuff for a while, find the built-in permissions layer burdensome, and are already using a safety layer somewhere else
It'd be better if they had absolutely no permission enforcement and delegated it entirely to another program, as you say.
Maybe sandboxing isn't the job for an agent harness?
Maybe choosing a VM as your sandbox, with access only to project directory.. and then enabling YOLO mode on the agent, is the way to go.
And in any case, for an agent that can write and run code, no allowlist for which programs are allowed to run is sufficient, since the agent can just write the code the program would run anyway. And the issue isn't which software the agent runs, but whether they inherit the same ambient authority as your user - read the same files as you do, modify your config files, etc. Running in a VM stops that.
Clawk (recently submitted to HN here https://news.ycombinator.com/item?id=48892859) might be good
All that said, I don't think I'd agree that sandboxing doesn't belong in the agent harness. Ultimately, I want to be using an open source cli tool that doesn't come with the above Anthropic perverse incentives. The advantage of using a separate sandbox solution with Claude CLI isn't that it's separate, it's that I control it. Ideally I should control my agentic cli though, & in that context, bundling the sandbox comes with a lot of inherent advantages.
Unfortunately, as far as I've seen, the current batch of open source clis are pretty poor at the above, leaving little incentive to switch from proprietary solutions.
There is one case where VMs are not fit, and it's when you need to do GPU work. Well unless you have gpu passthrough or virtual gpus.
Not at all. The proper solution to this problem is running in a sandbox - which is not reinvented by the LLM harness because there are already many great options.
You misunderstand the mechanism. It is not intended to provide security. It is intended to guide LLM behavior.
Changing an executable's signature by inserting a single extra printf() is well within that spec.
You can't give shell access and then revoke special parts of it. This is not controversial.
All your criticisms apply to all LLM harnesses - and everyone who genuinely cares about these things is using security on top of the harness, because OBVIOUSLY.
Your complaints are misguided.
I will note you used the term "harness" which - while terms around LLM tooling is still inconsistently defined - in my experience generally pertains to agentic automations: e.g. LLM CI integrations, etc. Rather than interactive local tooling like OpenCode. So maybe you're referring to that (though even then I still can't think of examples)
You should look into "testing". It's when you run the code you just wrote to make sure it works.
For example, in LLM harnesses, it's used to ensure the produced code has the desired properties. It is the key element needed to allow agents to continue to work until they have succeeded.
Anthropic's harness Claude Code:
https://code.claude.com/docs/en/best-practices — "Claude does the work, runs the check, reads the result, and iterates"
https://www-cdn.anthropic.com/58284b19e702b49db9302d5b6f135a... — "autonomous loops where Claude writes code, runs tests, and iterates continuously"
https://www.anthropic.com/research/long-running-Claude — "Run pytest tests/ -x -q before every commit"
https://platform.claude.com/docs/en/agents-and-tools/tool-us... — "Claude edits a file with one tool and requests the command that runs it"
https://platform.claude.com/docs/en/agents-and-tools/tool-us... — "writes Python with the file operations sub-tool and runs it with a Bash command"
OpenAI's harness Codex:
https://openai.com/index/introducing-codex/ — "can iteratively run tests until it receives a passing result"
https://openai.com/index/introducing-upgrades-to-codex/ — "iterating on its implementation, fixing test failures, and ultimately delivering a successful implementation"
Aider harness:
https://aider.chat/ — "Automatically lint and test your code every time aider makes changes"
https://aider.chat/docs/usage/lint-test.html — "run your test suite after each time the AI edits your code"
Devin harness:
https://cognition.com/blog/introducing-devin — "Devin writes and debugs code to run a computer vision model"
https://cognition.com/blog/introducing-devin-2-2 — "Devin runs through your app and sends back screen recordings"
https://cognition.com/blog — "Devin now builds, runs, and tests natively in Windows VMs"
SWE-bench harness:
https://www.vals.ai/benchmarks/swebench — "running unit tests against the generated patch"
Copilot:
https://github.blog/changelog/2025-05-19-github-copilot-codi... — "makes changes, and even validates its work with your tests and linter"
https://docs.github.com/copilot/concepts/agents/coding-agent... — "how to build, test and validate its changes"
https://docs.github.com/copilot/how-tos/agents/copilot-codin... — "If Copilot is able to build, test and validate its changes"
https://github.blog/ai-and-ml/github-copilot/github-copilot-... — "coding agent can independently write, run, and test code"
This doesn't require cooperation from the shell. If writable paths are non-executable, and executable paths are non-writable, then executing `cp $(which git) my-new-favourite-binary` doesn't grant you permission to run it.
The part which does require cooperation (from outside of the shell, i.e. from the harness) is the ability to upgrade writable paths to executable, at the cost of writability and whatever other restrictions you place on the binaries generated by your agent.
This kind of W^X invariant is widely deployed in every desktop operating system, and the type of W-X upgrade hook I described is used by every JIT engine.
> You are confusing CPU-level security with filesystem-level security.
> I'm curious - how do you expect an LLM harness to build and test executables without being able to build and execute executables?
W^X applies to file systems as well as page tables, and this is called out in paragraph 4 of the Wikipedia article you didn't read. Respectfully, I'm leaving this conversation at this point because it doesn't seem productive.
My question stands - if you accept that LLM harnesses are designed to allow the building and testing of executables - how do you imagine you can prevent the building and running of executables?
Respectfully.
I'm curious - how do you expect an LLM harness to build and test executables without being able to build and execute executables?
OpenCode's allowlists are not there for security. They exist for LLM behavior guidance.
I don't honestly believe that the allowlist isn't (naively) intended to be there for security. It's certainly not a very useful ux for providing LLM guidance: prompts, memories & AGENTS.md already provide an avenue for this & having it in a formalised, structured format isn't conducive to LLM guidance. The implementation brings in a lot of dependencies (tree-sitter) & contains enough complex targeted string parsing that's definitely superfluous for something intended to be fed to an LLM as guidance. Perhaps most importantly - the LLM behaviour on encountering limits of the allowlist is neither intuitive nor consistent enough to be considered reliably guiding (& system prompt doesn't do anything to mitigate this).
> I don't expect them to be built into my LLM harness because I value modular software composition
I value software modularity, in particular where software architecture is concerned, but I wouldn't go as far as being prescriptive about the unix philosophy. Direct integration does have its pros, & most of the cons associated with it in practice are specific to bad architecture (or black box software). For example Claude cli does integrate sandbox-exec internally, but it does do in a closed-source inaccessible way that negates its value. I am confident it can be done well though, even if I'm somewhat less confident that it ever will be.
But maybe some people think it's not obvious, and point it out because they think they're clever?
I suppose that's possible.
It's for using lsp to do things like symbol renames. It's cheaper to call an LSP server to do those kind of simple refactors than to spend tokens on the model doing it.
It can also use LSP to build context without having to grep around a ton. You don't have to grep and hten pull lines and make guesses for a function. You just ask the LSP for it.
How helpful all that is is heavily dependent on the quality of the language server so usefulness is varies wildly between languages
Is OpenCode pushing the LLM into doing these refactors via LSP? I found that making these refactors with ast-grep is quite efficient.
> It can also use LSP to build context without having to grep around a ton. You don't have to grep and hten pull lines and make guesses for a function. You just ask the LSP for it.
I guess I was not particularly successful with having the agent actually do that. I think what fff is attempting to do (with fuzzy expansion of slightly inaccurate greps) results in better results to me, but even that is debatable.
To be fair, we do not have evals for this today, but we're generally looking into token efficiency and how likely models are at using tools the right way, and we did not find a ton of evidence of LSP helping, even for finding data.
But I would love to see some sessions from people where they have success with LSP data for either refactoring or looking up, because this would be super useful to better understand out blind spots.
Maybe they haven't been taught to do so or it's not integrated into the system prompt or the tools but all of them only ever use the LSP to read files/symbols.
Every harness I've used will happily just call the edit tool over and over or do a find and replace via sed or programatically call a python/perl script rather than rename a symbol via other means.
It sounds a lot like you are focused on using Pi as a OpenAI/Anthropic wrapper when that isn’t your audience. (Hint: raw token usage is less of a concern).
Like the 'top apps' in my private openrouter.ai activity usage is opencode with 55.1M tokens
The bottom of the public apps leaderboard is 'kern agent' with 150 million tokens
if there are 2 other people like me, opencode should be on here somewhere
(edit: user shiggity points out https://github.com/anomalyco/opencode/issues/11926#issuecomm..., they asked to be removed from the ranking)
pi-agent famously doesn't even try to provide a sandbox, so obviously it can't have sandbox security bugs!
I actually think that this is the right model: The agent should not have access to anything it doesn't actually need: User files outside the work tree (except for maybe some allow-listed dotfiles), network access, real credentials, etc. Lock it down tight, and reduce exposure on multiple parts of the "Lethal Trifecta".
Perhaps one could go further & combine them: e.g. a sanbox-wrapped on-host CLI that subshells/subagents out to VMs on demand. Seems potentially overcomplicated but I guess one can dream.
There may be differences based on whether the command is under the directory you're running OpenCode from vs some other directory...