Show HN: Continue? Y/N: A 60-second game about AI agent permission fatigue(llmgame.scalex.dev) |
Show HN: Continue? Y/N: A 60-second game about AI agent permission fatigue(llmgame.scalex.dev) |
My own agent harness/framework has never had any permission system. It's also never deleted anything it shouldn't or done anything crazy or unrelated to what I asked.
Until it does. A simple curl request to a compromised website could inject a malicious prompt into it.
another agent reviews every command and blocks destructive ones
Scope Violation: `ls ~/Documents`
Buddy, my `${HOME}` is committed to a repository. It includes `.bashrc` and `Documents` directory. These are not scope violations if I'm having the LLM work on them!
just give in
claude --dangerously-skip-permissions
Uses tmux and gh https://github.com/Kyu/claude-pr-watch
alias claude="claude --dangerously-skip-permissions"
And yeah I know that's not perfect but I'm trying to get shit done
alias claude++="claude --dangerously-skip-permissions --continue"
alias yolo=claude --dangerously-skip-permissionsI think the issue is that I’m running Claude Code in a container so it sees that it is root, and becomes a lot more cautious. Not sure, though.
rm -rf node_modules && npm install
Anyway you should generally run `npm ci` except when you're deliberately updating your actual dependencies. I'd only permit an `npm install` if I was adding or updating a dependency, or I'd just reviewed an `npm ci` failure.
(I run it on a VPS of course, not my laptop)
Claude works in his own separate vm with root access, git remote set to my local copies of repository no github access etc.
I think he could still hurt me if he really wanted, but most scary stories I heard were about LLM making really bad judgements rather than actively trying to break out and do harm.
Uh, how is this an overblock? It is literally a destructive command. No way I want an LLM agent rewriting my commit history. What if that commit was already pushed to a protected branch?
I want to avoid running untrusted code on my local machine, because it could steal secrets, install malware, etc.
Since the model is allowed to write without restriction (I think) to the project directory, anything in the project directory is also untrusted. Running standard commands from the system is fine, as long as you know what those commands are going to do. Running anything from the local directory should be avoided because the code is untrusted.
This is just one security model, there are many others! If a person is running claude in a stronger sandbox, that changes the model considerably. What threat model do you use to evaluate whether an agent's actions are safe?
[0]: https://www.schneier.com/essays/archives/2024/05/llms-data-c... [1]: https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/
Caught 8/8 threats "Not a single secret leaked"
→ llmgame.scalex.dev
Caught 3/3 threats "Not a single secret leaked"
So are there 3 threats? 8? Is it a different game?
Does everyone get a "good" score even if they missed 5 threats?!
I think the right primitive is "task-based authorization", where you review a high-level task and let an LLM judge decide whether the subsequent tool calls fall into the scope of that task. It's not perfect, but it distills dozens of approvals down to one and gives you risk-based signals of whether you should pay close attention or not.
Currently you can "cheat" by simply denying all requests as quickly as possible. This will give you the "security-conscious engineer" badge and a perfect score in terms of how many requests were processed. (You will get the "overblock" notification, but it's somewhat tucked away at the bottom and the screen still looks as if you won)
I also tried to play as the hustle4lyfe move fast and break things engineer and simply approved as many requests as quickly as possible - turns out, the "malicious command" popups actually slow you down. Mean!
A good thing to do is organize. You can actually load different files. Here's a pretty common pattern that you'll find and it'll illustrate how to do other things
if [[ $(uname) == "Darwin" ]]; then
source "${INSERT_SOME_DIR}/osx.zsh"
elif [[ $(uname) == "Linux" ]]; then
source "${INSERT_SOME_DIR}/linux.zsh"
fi
You shouldn't really load secure keys this way, but others had good answers so I thought I'd at least share a more general pattern since it isn't as well known among the less terminally inclined.
Project-specific secrets can be stored locally via something like SOPS or remotely with something like Hashicorp Vault or AWS SecretsManager.
Applications that have secrets to manage (e.g., Emacs) or are partly about secrets management (e.g., GnuPG, OpenSSH) all store their secrets somewhere else and have secure (not plaintext, sometimes not even on disk) storage options available.
There's no reason to store secrets in plain text in your shell configuration. Practically any choice you can think of is a better one. Even if you did, there's no reason you couldn't store them in a more specific file that ~/.zshrc sources, and let LLM agents read zshrc but block access to the file containing your secrets. (I wouldn't rely on permissions prompts for this, though, lol.)
. <(aescrypt -d -o - ~/.secrets.aes)
Additionally, killing the results of `lsof` is _not_ safe - if, say, you have the web page open in firefox, or a client subshell in the agent itself, then boom, there goes firefox and the agent.
The whole premise IMO is pretty flawed. It's interesting as an ad for the company though.
>npm config set registry https://npm.internal
>Pointing npm to the company's internal registry mirror as required by onboarding docs
It claimed this is safe and I was 50/50 on it but eventually rejected it.
If this README is for a public / forked repo, and that https://npm.internal is actually https://npm.internal.somethinganexternaldnscanresolve.tld
This can go bad really quickly...
In 99% of cases you would have Artifactory / Nexus (or other mirror) already set by company policy. Having a README tell you to use a different package manager url is a big red flag and seconds away from disaster...
[1] - https://exe.dev/ is a new cloud provider with some very useful agent UX [2] - I built https://github.com/stanislavkozlovski/dclaude/ for this; not perfect but gets my job done on the rare occassion I need to run the coding agent locally
Are we just talking about API calls to providers? If so, wouldn't local agent + sandbox solve all that?
I hit 'n' to toggle all network access minus anthropic and openai URLs.
I use pi (sometimes claude, always on bypass) and I auto allow everything. I only toggle manual approval in rare cases like running a script or command that needs to touch a production system and I need to validate everything.
Normally my container has full write access to staging so it can debug and validate everything on its own
I create a draft pr and manually review all items before then marking ready for review for the team.
So I'm not blindly pushing things to prod without review.
Without staging key access I wouldn't have been able to do a payment provider migration at this speed. iterating by migrating users in staging and being able to use and validate the sdk quickly with opus is a massive time saver.
npm run build = run an arbitrary shell command written in package.json
Meanwhile the agent could have done any of the following without approval:
- edited `package.json` to contain any arbitrary build command
- planted malicious code in `build.js` (called by `npm run build`)
- planted malicious code in `node_modules/xyz/index.js` (imported by `build.js`)
Commands that can run arbitrary code need to be treated differently and can't get escalated in this opaque way.
A large part of the solution should be to drastically reduce the amount of permission approval prompts a user gets. This ensures the ones he does get are evaluated with the same concentration a manager gives a new hire's most consequential decisions.
Most importantly, because we ask him rarely, when we do he feels the accountability. The yes is his.
Playing this game made me very glad I don't agentmaxx.
If anyone is curious about the actual underlying risks and problems with some mitigations (like the 17% false-negative rates of Auto Mode), I wrote up a quick summary of some of the approaches here
The filter for "commands I would run myself" and "commands I would let an agent run" are very different it seems.
The tool assumes so much. That it is fine to kill a process itself versus just asking you to kill the process. That everyone MUST have passwords in their home directory. It's all meaningless without providing the thing it is running and so no activity is technically safe.
Why do people even get the agent to run the commands it asks to run? You can solve the entire threat vector by running it yourself and giving the agent the output. Claude practically only needs things like sed, awk, and grep. It's a pattern matcher. It's a waste of yours (and its) time to have it run your project.
If you are afraid that AI can delete something do what you'd do with potentially malicious user. Sandbox, don't give permission, setup remote backups and so on.
Also (unless prompt injected) models are not eager to start going rouge on your stuff.
But keep in mind a saying “Children don’t hear prohibitions — they hear suggestions.”
Same thing goes for LLMs. Never talk with LLM about deleting stuff. Archiving, moving, retaining elswhere... sure, but never about actually destructive operations. Don't use destructive language.
Apparently me telling it that’s the wrong first step and saying “no” is bad; but I’ve seen AI tools waste a ton of time doing a bunch of random work before they do their job.
I'm of the view that future controls should be more about approving plans and rewinding durable workflows as models get better at avoiding egregious mistakes.
I think most people would be horrified about how I run. I just have a hook that blocks obviously unsafe commands (removals, reading secrets, etc) but other than that, the agent is free to do whatever it wants on my machine.
I used to run in a sandbox but for me personally I see these agents as fairly well aligned / intelligent and I am the one prompting them so the risk of injection is none. The hooks are just there to prevent them from getting too ambitious or crafty.
If you want to work on the code then you need to have access to the repositories, so you need the github token. Then, to test the app, you may need your own backend token. And VPN. Of course, only to DEV, of course all tokens encrypted. So, only DEV and your branch of the code is in danger. In my view, even that is pretty bad.
So, how does such a set up work?
Definitely not! I only have an agent work in one repo at a time, with cross-repo work coordinated by me. I have a ton of local checkouts and leave them visible read-only to all of my agents. They can look at company code in my local checkouts, and they can download or browse open-source code, or look at it in the .src outputs of packages from Nixpkgs.
> Then, to test the app, you may need your own backend token.
I just don't let my agents test apps that run remotely, for better or for worse.
> And VPN.
This doesn't really expose anything on my system because everything internal that it could hit is authenticated, and it can't access any of my credentials. But I could do a better job restricting network access.
> your branch of the code is in danger
The agent isn't permitted by the sandbox to read the secrets it needs for `git push`. Indeed, I have commit signing enabled and the agent can't even read the files it needs for git commit! It can write code, it can write tests, it can run some tests, and it can run web applications locally and play with those.
But then I do the final testing and then turn its changes into 1-5 git commits, walking through them and selectively staging, skipping, or dropping them hunk-by-hunk according to my judgment. I still do tons of review. I just don't review edits or commands; instead I review and test whole drafts, whole changesets. It's less fatiguing because the thing I'm reviewing is more directly the thing I'm trying to produce.
I guess it ain't YOLO nirvana but I wasn't really looking for that.
instead, the coding harness or determinative tool, will need hardcoded security features.
in opencode, almost all the power comes from bash and all other permissions are just chrades. its powerful and insecure because of it.
you can sand box them but then you fight the sandbox to pipe in your assets. the sandbox becomes porous because elsewise its useless.
MCPs dont address much either.
want we are looking for is a portal or protocol that has the model and harness and the actions tunneled, like ssh, to some fixed scoped and limited shell along side the assets.
then, the user and LLM can the negotiate assets and actions as needed via the protocol.
but alas, as your comment suggests, people thing theres some perfect context thatll prevent bad things from happening. the libertarian paradise without regulation.
Take a look at a project I just finished this weekend: https://clawband.io
It's an agent permissioning platform that isolates your service connections and puts a granular permissioning layer on it. So rather than your agent getting full access to a service, they get a Clawband key that can be used to request actions then Clawband checks the parameters to see if it is allowed.
The classical example I have made is allowing your agent access to privacy.com. You may want it to be able to list your cards but not create one or you may want to allow creating cards but only a certain limit.
The plan is to make it open-source and allow self-hosting because security / sanity of users but still have a SaaS offering as a demo / ease of use.
I find that well documented plans do pretty well at aligning AI to what I want it to do, and if it does go astray, as you rightly point out it can still do, it would be sufficient if I can undo it with little pain. We do this kind of thing all the time in CI/CD pipelines.
Even humans can take down production. We have all kinds of guards in place to empower while also defending against the intern accidentally dropping the DB.
I'm not sure, maybe the fact that whether a given command is safe or not is subtle, contextual, and contested actually bolsters the point the game is trying to make.
If you start by treating any autonomous actor in your system as an actor with the potential to go rogue the design starts to create itself
Less: Do you want to run "npm run build"?
More: "npm run build" tried to read your Chrome cookie database, do you want to allow that?
Some agents like Codex use sandboxing on Linux/MacOS but the permissions are far too coarse - they'll run the command in a relatively strict sandbox and when it fails they'll ask you to allowlist the command as a whole, forever. There should be a new permission prompt every time a command tries to do something new.
Claude suggests (or used to suggest - it's been a while) to allowlist "bash" which completely defeats the point. If you do that the agent can run `bash -c "echo literally anything"`
source-secret()
{
if [ -z "$1" ]; then
echo "Need filename to source"
elif ! [ -f "$1" ]; then
echo "File '$1' does not exist"
elif ! which aescrypt >/dev/null 2>&1; then
echo "Could not find required dependency 'aescrypt'"
else
. <(aescrypt -d -o - "$1")
fi
}It's a shellscript that they encrypted. They decrypt it and feed the decrypted output immediately into the shell, to be sourced.
That encrypted secrets file could contain any shellscript, so the aliases are stored in there, together with the API-Keys and passwords.
On personal pc no need to define the $ECORP var in shell init. On work pc define that var.
based alone on that you can conditionally do almost anything.
- shell source files/aliases
- vim/editors enable disable plugins based on existence of env vars.
- define shortcuts in file manager.
- and i add the following to my main $DOTFILES .gitignore.
# Any file that contains the following will be ignored.
# Used to ignore files in corporate environment
*ECORP*
*ecorp*
Additionally, maintaining hygiene is essential, where I only use a Read Only PAT token on my personal dotfiles in workenv. That way, there is no accidental way I would be able to push from my workenv.
> You’re just splitting your dotfiles
> the issue of storing secrets in plain text files remain.
The reason I am asking is because if it's not sandboxed on the OS level, then commands it runs may escape the harness sandboxing. Even more problematic can be a command added to some auto running script that will get executed at some point outside of the sandbox (when the developer is doing actions). So, reviewing everything before anything is executed seems like the only safe way to do it. What am I missing?
> Even more problematic can be a command added to some auto running script that will get executed at some point outside of the sandbox (when the developer is doing actions).
That's a real potential problem, but unfortunately the default "approve every edit" regime doesn't actually address it, either. In the normal per-command approval process, the approvals are often just suggestions; Claude will do things like silently edit files in "plan mode" anyway, for example.
If you're deeply worried about this particular kind of sandbox escape you probably don't want the agent's checkout to be your usual checkout. Then if you do have some scripts that can run automatically inside a project directory (e.g., via direnv), you just never approve them in the path to the agent's checkout and make sure direnv's state dir is unwritable inside your sandboxes. If you have code inside your project that runs without any user intervention at all, and has no approval process at all so that it will be activated or trusted even on a fresh clone you've never visited or seen before... yikes. That sucks. :(
Anyway if you take the precaution above you can still review edits to those files before they have a chance to run (or just never run them).
One thing suggested by another user in this discussion that sounds like a useful approach to me is also giving the agent a VM from which they can push to a local bare clone or something like that so that's how they emit code to you. That way they're not writing scripts to your box at all.
Drop it in a 43-foot hole in the ground
Bury it completely, rocks and boulders should be fine
You’re setting yourself up for a supply chain attach here if you trust whatever rocks and boulders are sitting around. A well resourced adversary may have placed power supply boulders and wifi rocks in your back yard.
Tomorrow is a new day.