Sandbox: Run untrusted AI code safely, fast(github.com) |
Sandbox: Run untrusted AI code safely, fast(github.com) |
https://github.com/liquidmetal-dev/flintlock
There's lots of others. I'd love to see a proper comparison somewhere.
I had a brief glance at running firecracker VM's locally as that sounded interesting, but it doesn't seem too easy.
Does anyone know of any good solution that improve the UX of that (running some firecracker VM's locally)?
It uses KVM directly on Linux and Virtualization.framework on macOS, with a builder API for VM configuration. For AI sandboxing specifically, it has a higher-level "sandbox" mode with a guest agent for structured command execution and file I/O over vsock. You get proper exit codes and stdout/stderr without console scraping.
Also supports pre-warmed VM pools for fast startup and shared directories via virtio-fs.
I'm planning to support OCI images, but not sure if that's important to people. I typically just build my own root disks with Nix.
I want to have a "container" (used in the conceptual sense here - I'm aware of the differences between container and other solutions) that I can let an AI agent run commands in but is safely sandboxed from the rest of my computer.
For me this is primarily file access. I don't want it inadvertently deleting the wrong things or reading my SSH keys.
But the way the agent uses it is important too. They generally issue the commands they want to run as strings, eg:
bash ls
sed -i 's/old_string/new_string/g' filename.py
In terms of UX, I kinda want something to paper over the inconsistencies of the different tools I need to use to set up the network etc. (Kinda like the `docker` CLI tool).
When I looked at it the first thing I thought was "the tun/tap setup seems fiddly, and I bet I won't leave things in a consistent state (note, I just glanced at this blog[0]). The copy on write filesystem stuff looks cool too, but also fiddly.
The more I think about it the more I just come up with "just docker but VMs".
[0] https://harryhodge.co.uk/posts/2024/01/getting-started-with-...
1. coderunner - https://github.com/instavm/coderunner
Also if it is agentic, why is it less cloud based than eg Claude code? Are there LLMs running locally?
It's not agentic - agents can use it to execute code. Those agents can be powered by any LLM including local.
although this is self-hostable on gcp, they can get quite expensive due to the machines used. cheapest vm with nested virtualisation on gcp costs about $60/mo. on aws, you'd have to go with bare metal, which can cost you a lot more.
i think the next best thing for sandboxes is, "vm as a library", atleast for personal/small scale workloads.
The issue with using raw VMs is you want fast startup. If you are running hundreds of pieces of code per hour as you develop, or have 10 or 20 agents running simultaneously it's much better to have something faster to start.
This uses Amazon's Firecracker on GCP to provide that.
AWS has something similar for its own Agent framework.
ie "Here AI, call this function -> local_exec(commmand_name, {param1, param2, [etc]})" to execute functions.
And you'd wire up your local_exec() function to run the command in the container however you choose. (chroot, namespace, ssh to something remote, etc)
It's unclear if it is a container manager, or comes with a LLM Agent built in. These are two separate concerns and the README makes it very unclear how to use one without the other.