The Brain Sandwich(terriblesoftware.org) |
The Brain Sandwich(terriblesoftware.org) |
So such posts will make as much change as "trade, but be ethical" would make on a late 80s Wall Street trading floor:
"Before you tell the agent to start building anything, you go there and read and understand it first."
Yeah, not gonna happen. 1 in 100 might do that.
- AI generates too much code that overwhelms humans
- AI deskills users concerning the gritty details of coding by having them not do it themselves
- Having most of one's co-workers become meat proxy will put pressure for others to convert
- Lack of effort or expertise invested into codebase leads people to care less; caring less -> not caring about optimal outcome -> not expending brain power on problems we don't care about.
If one can't understand the monstrosity that was built up over time, where the code base is enormous, millions and millions of lines and components, all with a few buggy corners that the tests didn't touch, and a human didn't have the insight to write a test that understands and verifies the corner cases, it will be like a magic machine that we can't change because we don't understand it and we can't afford years of effort. Some may hope their LLM will be able to understand it.
We'll have things that are buggy in ways that become very difficult to understand or fix, and no one can afford the effort. Remember when we had all those huge old programs from the 60s or early 70s, before there were tests maybe, or unit tests. Code written at a lot of human effort, but before the ideas of software engineering made it more tractable to build large programs reliably, testable.
We are creating a new problem for the people that take over our new code, just like the "vast for the time" 1960s huge cobol program can't be changed.
Humans have been building stuff too complex to understand for a long time.
But yeah, AI is going to accelerate this…
If we assume that AI coding is just beginning- we can assume that humans will create systems to help Manage the increasingly complex interactions that machines will code - ie we are still early like when we first introduced the industrial revolution to the peasants (which led to decades of chaos- for sure)
But Just like a 787 is not flown by hand either - doesn't mean anyone is qualified to autopilot it
Complex engineering systems will require sme humans to help manage the underlying complex interactions in way that is comprehensible to human intelligence
No one except for a smaller group of specialists for example these days does much understanding of assembly or compilers.. but we build complex SaaS apps all the time without that knowledge
1. I wrote a status-update without actually looking at the status.
2. I made a proposal without really knowing the pros and cons.
3. I provided you information without really knowing what your question was.
If the software makes money because it's expected to work reliably, every edge case matters. Web games aren't exempt from engineering challenges. In fact, the more trivial the problem seems on the surface, the more likely it is to have edge cases.
Bugs are always prioritized based on how big of a deal it is for the customer. They don't like to wait and they rarely give second chances.
AI doesn't fit into any dev workflow except greenfield or analysis. Everything else needs a lot of close human attention and human discussion. AI doesn't read minds.
Pedantic addendum: this is an AI sandwich, not a brain sandwich.
Besides don't most people refer to the croissant in substitute for sandwich "ham and cheese croissant"?
I'll try using Brain Sandwich as guidance for less technical folk.
But the Fall Festival just ended, it's the perfect time to bring it up!
When I was in charge of companies, I understood the technical tasks, but delegated understanding of things like tax law. This was okay, but I was definitely at a disadvantage compared to companies that could better optimize capital allocation around understanding of tax law.
AI means more humans in every organization are having to learn how to be managers/leaders of these agents. Those who actually understand the work they delegate will maintain an advantage over those who do not.
The sentence is maybe ambiguous, but it sounds like it's saying to not use the AI at all until you understand what's happening in the area that needs to change.
I don't agree with this as you can spend quite a bit of time digging around to first find the correct place which needs to be updated and then more time reading the code (likely in multiple files or places) to build up a mental model of what's going on.
I typically have more than one AI conversation on the go at any given time, so I'll always start by asking it to make a plan to solve a problem and while it's busy exploring to make a plan I review some other AI conversation's work in the meantime, when the AI's plan is ready I review it.
If at this point there is anything that is unclear to me about its approach to implement the plan, or I don't already have familiarity with the part of the system it's proposing to update, I start questioning the AI on how the system works. I will scrutinize the AI's plan until I'm absolutely sure that it's going to implement something that is sound from a system design perspective and as a senior software developer with architecture level experience I am capable of making such judgments.
I'm not sure what it might be like for comparitively inexperienced developers, but I would expect that AI's capacity for infinite patience allows them to ask as many questions as they need to get a clear mental model of what's happening, so if I were mentoring such a person in this age of AI I would drill into them on how to interact with AIs to ensure they have this clarity before the AI starts implementing anything.
The difference between vibe coding and highly accelerated agentic development is having the expertise and judgment to ensure you're building properly sound software.
1) While pointing the agent at the task, poke it with annoying questions until I understand the problem and its context well enough to explain it.
2) Have it write the solution, help it work past problems it encounters during testing, then go through simplification, performance and correctness reviews.
3) Have the agent draft a pr, then poke it with annoying questions about the PR until I understand where we’re at, and why we don’t need to do more work. Also try to rewrite it into readable prose instead of Claud-ish while doing so.
#2 and #3 can repeat quite a bit, depending on how stubborn the problem is.
Yes, but the amount of work needed here is overstated. With good orchestration/prompting in the task[1] (and token budget), the work on this end is significantly reduced.
… and then the brain drain issue returns.
[1] https://claude.dev/blog/solving-code-review-at-anthropic/
(Before someone asks: "not using it right" could mean a lot of things, but intentionally constraining its search by telling it how to do everything is one example. Or by not giving it a way to hill-climb: no tests, no iteration on a solution, etc.)
These types of people will get the least out of AI as their ego prevents them from letting it outshine themselves.
AI is reasonable at reviewing code, with only a 50-60% false-positive rate, and its persistence at broadening scope often finds things human reviewers would miss. It's still actually still pretty shitty at writing code, though, again, with enough ground truth and persistence, it'll often generate something that's a functional turd. It will sometimes come up with better suggestions if you prompt for it, but it needs a ton of hand-holding at the moment. And usually the suggestions are still overly complicated messes that you can trim down to 1/10th the size, while keeping the core idea.
1. Your brain, and then
2. John von Neumann, and then
3. Your brain
Explaining a bit more… Before you tell John von Neumann to start working on anything, you go there and read and understand it first. And when John von Neumann brings you the solution, you will know if it’s right, or if it’s too much, or if it’s completely solving the wrong problem, etc.
>AI, and then
>Your brain
I'm sorry but this is not a "brain sandwich" this is an ai sandwich...when a slice of ham is between two slices of bread do we call it a "bread sandwich"?
> The rule of thumb is: if you can’t explain what’s changing and why, it’s not ready for someone else to take a look
I have observed firsthand that the metric or "understanding" can be irrelevant to management. When delivery dominates, understanding can be deferred to future agents who will gladly consume your codebase and convince you they grasp it, and further, that you do too. If I invest substantial effort in my PRs only for them to be /review'd, does it still matter? Perhaps reviewers should practice brain sandwich as well.
The worst offenders seem to sit at either extreme: fully AI firms that treat understanding as a human weakness and prioritize outcomes alone, or large corporate teams building internal CRUD apps whose middle management merely pretends to value good engineering to satisfy finicky senior engineers.
Not a criticism on the authors work at all, I like the idea (and I don't think they were trying to solve anything to do with work culture), but ultimately if my company doesn't find my practice valuable, I can't easily justify taking it on. Especially if I have to sell to them that I'm going to be spending _more_ time delivering. Fun times!
This is anecdotal of course, but at least at my company it's getting a bit easier to advocate for humans to do the thinking and understanding.
In this case, we're the peasants.
And of course, there are things like build systems (CMake, ninja, blaze) where I know the basics, but I've never been an expert and don't care to be. Before I'd offload build features to another team; now I'd offload it to an AI. It's not something I need to be an expert in. I just want builds to work well.
Web games don't make money because they're expected to work reliably, people play them on impulse. If it breaks 1% of the time, that's hardly a problem at all. If a kernel breaks 1% of the time, nobody would use it ever.
A raytracer demo doesn't need to be engineered to the same standard as the Apollo onboard computer software. A $20 paint app doesn't need to work as reliably as banking software that handles millions of dollars of transfers every day. All software exists on a scale from trivial to critical. It's like saying a toy drone needs to be designed the same way as a F-16.
Why wouldn't a game dev do the same? It's pretty common for initial game releases to lack features.
Ie you are making the mistake of assuming the tech is static when it most assuredly is not
That difference is enough to unlock AI coding for so many real life teams
Sometimes in review I'll be surprised that my understanding was flawed and the AI subtly misled me during the discussion, but that's super rare and that's honestly still helping me learn and understand.
Frankly it's no different than if I were pair-coding with a teammate who happened to type 25,000 wpm, and who never takes a day off.
They are not going to do a good job.
Software engineers used to mock non-coding architects for good reason.