Towards Self-Driving Codebases

Towards Self-Driving Codebases(blog.detail.dev)

35 points by wilhelmklopp 2 hours ago | 16 comments

    > It’s actually fine if agents make a lot of boneheaded mistakes. What’s not ok is if they keep making the same mistakes.

I worked in life sciences for a bit. There is a process in clinical trials called corrective and preventative actions (CAPA). You'll also find this in other areas where failure tolerance is low (e.g. aircrafts).

It's simple: when a mistake happens, you run you CAPA process and determine the root cause and the correction to the process that allowed the mistake to happen in the first place. This is something most eng shops do not have the discipline for since it requires some diligence.

Should it be fully agentic? Should there be human intervention here to approve the CAPA? Open questions to be answered.

skinfaxi 19 minutes ago | |

> Should it be fully agentic? Should there be human intervention here to approve the CAPA? Open questions to be answered.

I think a point we are grappling with is, what necessitates human intervention, like philosophically. Is it accountability? I was thinking about this in terms of code review and it's not like we would fire someone if they broke prod, so at what point do we need someone accountable. Maybe it's for approvals for certain classes of risk (like those associated with actual harm to life). It's also not lost on me that many human systems lack accountability.

drob 32 minutes ago | |

Agreed that domain experts should be making these kinds of decisions, and the right way for an agent mistake to get caught is going to depend a lot on the cost and probability of the mistake. My only point here is that the agent side of the system needs to learn. The software production machine needs to improve over time.

aerodexis 12 minutes ago |

Makes sense to me - I see it going down two separate routes in parallel:

One is to take the human out the OODA loop for cyber defense. Servlet libs (for example) are going to become fluid, self-modifying things w/ contracts that operate much differenlty from how they do today. The engineering practice around these things will need to change.

The second is that UX will be self-modifying. Just like how pi can modify itself, I can see this being a general practice for user-facing applications. Perhaps a text-box in which users can describe tweaks in can request changes to how the application functions. Engineering an application will focus on modeling the non-negotiables of how an application works, and providing the correct primitives for user-driven LLMs to modify software on the fly, as well as track and rollback changes. There'll also be funsies around how to ensure that ads get delivered regardless of the user trying to get rid of them.

smallmancontrov 3 minutes ago | |

> fluid, self-modifying things w/ contracts

This smells like worm food. Those contracts had better be airtight.

geooff_ 5 minutes ago |

If you have a logo in the top left corner of your company blog the logo should route to the home page so I can see what product you're actually building.

wrs 23 minutes ago |

This would be more convincing if there were a simple stat on the home page of the dispositions of the reported bugs (fixed/no-repro/won't-fix).

As a random sample of one, I looked at one of the bugs this reported on Tailscale (first thing on the homepage) [0], and the pull request ends with "Apologies for the lack of due diligence here. I'll go ahead and close this out."

[0] https://github.com/tailscale/tailscale/pull/17843

drob 16 minutes ago | |

We should publish some stats. The fix rates on bugs are around 45% baseline, increasing over time as we learn from behavior within a given codebase. The merge rates for the codebase health work we do (e.g. deleting dead code) are very high, at least 90%.

rrook 1 hour ago |

My bet is that we'll see a second layer of harness emerge, as self-driving codebases become the target. There will be an application facing harness, orthogonal to the agent facing harness. The app harness will represent the software factory that is emergent for the specific application being developed.

Anyway, here's mine, still wip:

https://hale-lang.org/docs/dna/

https://github.com/hale-lang/hale/issues/690

cyanydeez 49 minutes ago | |

I suspect it won't be a harness, but just a more specific LLM trained in the universe of user-selected context of vetted resources.

Why? Because LLMs are always going to be dumb when they're trained at scale. Their ability to speak software diverges from their friendly user input layer. A harness won't overcome that, but an LLM saddle ontop of a larger model would provide the type of feedback loops you'd want to look into.

I don't think you'll find two deterministic systems will produce much.

rrook 2 minutes ago | | |

I think that dissolves the "self driving" distinction, though? The mechanisms for driving the codebase must be present in the codebase itself. Otherwise its just a regular out-of-band development process.

verdverm 17 minutes ago | | |

fine-tuning may be a more scalable approach to LLM personalization than sending all the same context to two LLMs

I'm working towards both in my homelab to see which works better with little qwen

drob 35 minutes ago |

Author here, hello! Happy to expand on how we're thinking about this if any of it is unclear.

We wrote this post as part of a launch, which you can check out here: https://x.com/danlovesproofs/status/2095182189499711759

sroerick 40 minutes ago |

This article hints at it, but I think the blocker is environment.

How can you self drive an app on windows? There's no clear UI framework, design pattern, nothing which can bullet proof your app.

I bet you'd have better luck on Plan 9.

Remove the choice. Make frameworks which have limited options which are usable by default.

writeslowly 27 minutes ago |

I suspect that if you're not careful with agent memory it creates a danger of agent-driven cargo-cult behavior. I've watched this in my own ad-hoc agent loops where it starts with something basic, like the first agent tried to run some gigantic dependency inspection command and OOMed the local JVM and eventually recorded a workaround (to enable it to run gigantic dependency inspection commands...), and by time I get a few more agents into the loop, agents have written entire paragraphs about testing and validating local dev environment memory configurations that are mostly irrelevant to whatever is being worked on.

In general I've seen other issues like this where small errors and irrelevant comments in the codebase spin out into larger problems that consume annoying amounts of time/tokens. Maybe Anthropic and OpenAI don't notice this because they're in an "infinite monkeys with typewriters" scenario, but it's noticeable to me when the agent in my CLI has been spinning for 15 minutes contemplating irrelevant details

memjay 6 minutes ago | |

This happens way too often in my experience and I tried a lot of things. If the agent has access to an existing codebase it will always try to stay as close to it as possible. It doesn’t diverge from the initial direction the project/code went into, even if it’s already clear that this direction goes nowhere. Somehow agents aren’t able to detach themselves from existing code to come up with completely new solutions.

Sometimes I feel like it would be best to only give the agent access to clear API boundaries (say public interfaces to certain modules) and let it work out a new system from scratch given the expected inputs and outputs. Then plug this independent solution into the system. Of course it can still overfit these interfaces but it’s less than having access to the entire codebase.

Thinking about it, maybe it’s possible to let one agent extract those interfaces, then use them as grounding for a new session.

One less nice way of achieving the same is to tell the agent once in a while that it should think completely from scratch (from first principles). But this relies heavily on instruction following in the reasoning part, which sometimes works and sometimes fails.