The end of "Just ask Sarah"(simme.dev) |
The end of "Just ask Sarah"(simme.dev) |
Posts like this frustrate me. Not because of what they ask, but because of what they incorrectly assume. They assume that documentation can provide enough context, and that human knowledge is not needed.
Every bit of written documentation can and will be misinterpreted. And perfect clarity is impossible. A well-written ADR does not eliminate all ambiguity, because there is too much cultural context around the writing of the ADR that attempting to read it from some other cultural vantage point leads to bad assumptions. We can find this basic lesson from reading law (2nd, 14th amendments to the constitution), history (what did happen after Muhammad died?), philosophy (what in the world is Plato's cave talking about?), or theology (how should we translate Ephesians 5:22-33 and what does that mean) outside its original context with other people.
Just writing things down and thinking an AI is going to later perfectly understand what the intent of the author is... patently ridiculous. I do not intend to dismiss the idea that we should probably document more, but the idea that the AI can just take our documentation and competently understand all the decisions represented in them is ludicrous.
- I'm writing this service even though team X has built the same thing, because my team lead doesn't trust team X since the last time we depended on their service 3 years ago, they had a major downtime that screwed us up big time
- This service is using AWS Lambda simply because I think it's cool, despite the fact that the company has a dedicated team running k8s stack with argocd, argo rollouts, KEDA, etc for the entire company
- Service Y is written in this particular way because it's a service that is shared with another team that came from a company that was acquired, and they wouldn't use it unless we write it this particular way, and making the top execs happy is more important than dealing with a small tech debt (this is probably true)
But no one is going to write these in their RFC. But Sarah knows.
It's funny actually, because I fully agree with your reasoning. The only part were we differ is whether that's assumed, or even implied.
No documentation means running fully on tribal knowledge, or institutional knowledge if you prefer. Even if you capture your intent, imperfect and incomplete, in as little as 2 paragraphs, you'll get durable recorded memory, and intent you'll be able to reference. It does not eliminate ambiguity, but it adds framing, direction, and friction.
The examples are great, and they serve really well to prove another point that I intentionally left out: writing is not a one-shot activity. Documentation is living and should be treated as such. Unless it receives proper care continuously, it will wither and die. That could very well be the topic of a future post!
Thank you for reading and for providing thoughtful feedback!
It feels hard to recognize that you agree with me from the following paragraph of your post > This is similar to asking Sarah why the team went for A over B, but without the imperfections of human memory, and available to agents as well. When an agent reads it, the decision isn’t an arbitrary historical fact - it’s the conclusion of an argument the agent can now evaluate and extend.
I definitely agree that human knowledge is imperfect, and that documentation is needed. But I just cannot get around the fact that the following sentence, out of context, can refer to football, American football, and basketball : ‘I passed to the center, and we scored’ and without more context, each of those can have multiple meanings even in each of the sports I named.
const hello = ‘world’
Is valid in so many programming languages that need more context.
No, the agents don’t ask Sarah, but maybe they should. And maybe I’m on this soap box because the thing is that I write internal line of business software and the technical stuff is easy. But when Sarah is the inside sales manager or the industrial engineer a continent away who speaks a different language, the reality of adequately capturing context for decisions that others have made that our software has to implement, and in a context rich enough for documentation to be succinct, is enough to make me frustrated with articles like yours that hand-wave that into ‘trivial to capture in documentation.’
All of this to say, tribal knowledge is valuable and rarely well captured. Even experts organize conferences and lectures instead of just sending each other papers. Written words is better than nothing, but knowledge is better transferred from the person that has it. If you lose that person, the only way forward is to train another one to replace it (either intentionally or not).
Whether bulk or terse, highly precise with words or highly nuanced in how it's communicated... I don't think any of that gets you to a place where the documentation is a substitute for asking the person.
People are trying to do this with meetings as well and certainly they help but code that's written plus meetings plus an architect like person yammering on endlessly about the nuance that went into it still is often not enough to capture the detail in earnest – and especially not in a way that won't be misinterpreted.
Perhaps if AI becomes truly superhuman in all of the relevant areas to the point that it makes the decisions just as well as the person in the chair... _then_ we might solve this by having it instantly pattern recognize the solution and the why, but until and unless we reach that day, I think what you're saying is very true.
At the end of 30 minutes I either understand how the project works at a high level or know enough to know I'm not going to be improving anything with this project, ever.
Yes, humans (and human languages) are flawed and lossy.
>A well-written ADR does not eliminate all ambiguity,
True: no docs can ever eliminate all ambiguity (on a decent sized project at least).
But this entire argument seems to be "letting perfect be the enemy of the good". Documentation doesn't have to be perfect or 100% unambiguous to be useful.
My main complaint is that articles like this lead people to think they they can avoid those three problems, without understanding why there are three.
Very, very few of these organizations have ever known, and fewer still have ever cared, about their Sarahs.
This isn't the end of Sarahs. Sarahs have never had their time or place beyond immediate teams, many of which have used Fight Club rules when it came to their Sarah: Never talk about Sarah, especially not to the boss. Other, non Fight Club rules: When Sarah is away, cover as best you can. Change jobs before Sarah retires. It is not the end, because the time of Sarahs never began.
So I agree with ";dr" comment, but it would apply had this been written by a human, by AI, by a super-intelligent shade of blue, or a small furry creature from Alpha Centauri.
the agent has the operator, and the operator has a sarah
i have spent a lot of time answering colleague's agent's questions as a sarah.
my thought has been to try to commit any of the answers to those questions into the relevant codebases so that they become findable.
same with anytime i have to give the agent extra context about some code. The end goal being that as much knowledge as possible is out of my head, and put nearby the code where its most useful
I just think this is entirely wrong. Oral tradition is valuable because it's flexible in a way that written tradition struggles to be. Just a handful of oral-tradition decisions I routinely see that could never be written up as persistent documentation:
* The CEO said X is our top priority, but we think Y is more important and we can do it without compromising too much on X, so we're going to do both.
* Team A has a track record of quality and success, so their decisions are subject to less review and receive more deference
* Team B is sloppy and makes a lot of bad calls, so we don't trust their judgments when doing so might lead to an outage for us.
The next two don't have to name names, but absolutely can name practices and privileges. Do these things, participate this way, get bonus. Do other things, no participation, and no bonus. Be written-clear about the expectations and consequences. And then hold teams accountable.
It turns out wrapping the patch with these things makes the patch better. Plus, you have docs. All this goes in the PR.
Humans think docs cost/waste time. Agents not only don't care, but code better when doing SDLC (because once you get rid of the cult ceremonies, the core principles work). And so do humans.
Maybe Sarah can write it down, and maybe so can anyone that gets an answer from her, and maybe so can the LLM.
This is such a toxic combination, becuse it requires significant people skills to get out of the "the kid who learned everything here and is grateful for it" and get proper respect as a professional. At some point the only option is changing jobs. I've seen companies matching your offer, finally realizing you actually have value in the market, but don't count on that, don't bluff.
I too ended up as the "go to guy", partly because I had a lot of enthusiamsm for my new job, and partly because the talent pool there wasn't very deep (or maybe they were smarter than me). It's fulfilling until it becomes unrewarding, I had to move on after almost 5 years. Still did consulting for them ocassionally for a couple more years.
If you have golden handcuffs and multiple promotions out of being Sarah, what is it costing you?
That sounds exhausting to say the least.
It’s very easy to turn into the Sarah - or the Brent if you prefer the Phoenix Project analogy. As exciting as it might initially be to be the go-to person, it’s also, as you so elegantly put it, “endless work, just enough authority to do current task, not enough respect/authority to solve the symptom”.
Best wishes! I hope you manage to turn it around.