Programmers will document for Claude, but not for each other(blog.plover.com) |
Programmers will document for Claude, but not for each other(blog.plover.com) |
The problem I have had is other developers expecting me to maintain documentation for their tools. To the point that they wage stupid inter office wars because they don't want to learn a command line utility with 20+ years of documentation itself.
In engineering of all kinds (or at least the ones I'm familiar with), nothing really beats calmly sitting with your thoughts, stating a problem, then getting up and walking around while you think about it, then sitting back down to write down a possible solution, and then asking colleagues to read it.
On the one hand, I also feel like "come on, couldn't we have done this earlier?"
On the other hand... the costs of the docs have decreased. Simply firing a frontier model at your code base doesn't always produce perfect docs but it's a heck of a good start. I do recommend some tuning in the request, e.g. I like to explicitly ask the AI to document data flow rather than the usual list of "here's this component, here's this component, here's this component", but it's pretty easy.
And the utility of docs is now much higher. I really just recently moved into the classical "architect" role and in some ways I'm glad it wasn't much sooner, because my GenX cynicism tells me that nobody ever reads the architecture docs. OK, OK, sure, technically nobody is a bit too strong. Sometimes, some particularly intrepid or conscientious souls surely read them at some point. But from my own experience I could count on being able to hand out API docs, structure docs, flow docs, and their primary utility was that when someone tried to deflect responsibility with "but but but they didn't provide any docs" they couldn't, because I had. People eventually learned to stop doing that because I always provided the docs because I saw that coming. And they made a great background on the shared screen as I had to walk someone through the entire thing in a meeting anyhow. They were more a really specialized meeting transcript than something I could provide in advance and expect much out of.
But now, if nothing else, AI will read the documentation. I can tell people to pull it in, and while it doesn't mean all my problems go away, there is now a much cleaner path for me from "writing an architecture doc" -> "lines of code in somebody's repo" than there was pre-AI. My architecture docs are now somebody else's prompts. The utility of this sort of documentation skyrockets compared to the old days.
So, when the costs decrease and the benefits increase, it isn't a surprise that suddenly, it's easier to get some of these things done that we "knew" we needed for a long time, but now with the new cost/benefit ratio can cross the action threshold.
2. Claude will actually read what is written (well parse for autocompletion, not actually "read", unless you are under AI-psychosis).
A cheese-grater Mac is not a door stop either, yet look at it go.
Will people ever let go of being hung up on how exactly an LLM produces text, or do I really have to keep listening to this shit for the next several decades? As if being a program didn't already prohibit them from reading to begin with!
Tell me how you don't think a plane should be characterized as "flying" unless one is in "avionics-psychosis" or something. Surely you can appreciate how this narrow requirement for avoiding anthropomorphism and metaphors is entirely performative, and that engaging in them is in no way a sign of any mental illness.
And I get the performance aspect behind it, people wish to reject the metaphor to tear into the thing even that way. Or they come from a religious or spiritualistic background, and find the mere comparison insulting. But fuck it's so cringe. It's not news, and it's not insightful. Nobody who's ever had them shit out a page per second via like a dozen subagents, or utilized them being stateless and seed-bound, has any actual illusions about them being anything more than "just text generators". If they nevertheless assign intelligence, understanding, or similar traits to them, then clearly they simply don't agree with the anthropocentric philosophical insinuations performed. It's not some mistake or psychosis. Come on.
Documentation is firstly for yourself, secondly for your coworkers and/or users, and only last for AI. It is the art of writing and revising it that strengthens+questions+maintains one's understanding, so it doesn't make sense to have an AI write it.
The point of self-documenting code isn't to get rid of the documentation. Instead, it's to integrate it into the code. This fixes the two biggest problems with documentation. First, that the code will often be updated and documentation left behind, making it useless. And second, that English and code are intertwined in the same document, making you constantly switch mental contexts of how you are reading. So no, I don't consider developers who write documentation to be inferior. If anything, I value documentation even more, and the purpose of self-documenting code is to make the documentation better, not get rid of it.
With Claude, the documentation you make for Claude is just fine as it doesn't run into any of the above two problems. The documentation isn't being left behind, because you add to it instead of the code. And it's not intertwined with code, because you are just writing English for Claude, you are not writing code in-between.
Claude emits garbage, so documentation will be garbage as well. It doesn't matter, because everything will be rewritten and tokenmaxxed anyway.
In other words, the blogger now has a job where Claude is foisted upon him and he toes the line.
Uh, you needed to do that for humans too. You just didn't. There's a reason everybody scrolls to the bottom of man pages ASAP.
Unstrutuced slop is no better at best, and much worse at worst.
I haven't had too much problem with information in summaries being wrong, but there have been times the LLM will miss the most important details. Then when you tell it, the response is "Nice catch!" or something like that.
When you say "you haven't had much problem" one can only assume you're _not actually reading the output_. In fact, like most things in modern times, one has to assume you arn't actually reading the output. You're skimming it; you're finding what makes sense and extrapolating that. This is the 70%.
The problem with non-deterministic models is that the output can't be deterministically assessed. You're harboring a delusion that you're getting real good output.
Most likely you're doing the baby extrapolation: you make it do a small, tightly scoped project and it's does 99% right. Just like a baby doubles in size in a year. Extrapolating, that baby will double again; but it doesnt.
Your human compensation limits does not extrapolate to the size and knowledge that's fed into the model and the context it extrapolates.
When someone created a CLAUDE.md, then changed some stuff around and when I later had to touch that repository my claude was hallucinating classes, functions and architecture that was already long gone!
I just deleted the CLAUDE.md, since I had no mood to "fix" it.
LOL. It didn't even cross the author's mind to consider reading the notes himself to decide if they make sense.
AI is sending us down a path of anti-social behavior. It will be bad for us, of course, but AI is only as good as it is because it was able to be trained on info from github, stack overflow, etc. Without socializing interesting info, humans and AI will both suffer.
I used to envy and take inspiration from other workplaces where good [but not necessarily good-only] writing was respected; where a pre-read is really read before the meetings, thoughtful comments were made on it, etc.
AI workflows have obviously simplified documentation generation along with the code, but we had to work on our product/engineering practices to generate meaningful documentation, and not just vestigial/temporary documents in the process. On this particular point, we've made positive progress lately.
The wealthy/owner class once again consume all of us -- here through AI -- because we cannot agree to work together.
You can step off your high horse now.
1. Immediate better output from the machine OR...
2. Being sidelined for career promotions because you spent so much time making sure documentation was accessible while everyone knows they can ask you instead of reading it, and you will answer.(reference: https://knowyourmeme.com/memes/im-in-this-photo-and-i-dont-l...)
I don't view it as something I'm writing for someone else anymore.
Ultimately, I think that's helped me write better documentation.
Writing for an imaginary audience is typically harder than writing for a concrete one, and things you might think "someone else" might need to know - they either 1) don't, or if they do 2) won't read your docs anyway, most of the time.
And this is why LLMs are great for looking up docs. It makes any docs work for any one as long as the information is present in the doc.
* Sometimes, this prompt is enough for them to find the answer.
* Sometimes, they tell me a spot that makes sense to them, and I make it have the answer. (Maybe just by adding a cross-reference.)
* If they refuse to look at the docs, I can't help them.
You got very good answers to any original questions, though you should start by showing where you searched for answers already.
They hated "do it for me" kind of requests that usually looked like somebody asking you to do their homework assignment. I even got called out on it once, but could happily reply that I'm actually just messing with my system as a hobby.
One time I had a "do it for me" request.
I've run a sed command which appended sth.bak to every file of some type on, but accidently made it execute on all files on the system. They quickly gave me a one liner to fix my machine (a VM, but it took long to set up). However, when after fixing the system, I asked for explanation of the xargs command that was used there, I instantly got sent off to the FAQ with a number.
My boss was looking at them, but even the simple ones he was pointing claude at it and asked it to make a document explaining it. Then he'd send me the document and ask me to check if it was accurate. I added a line to the last page "this is an ai summary and may contain mistakes. Use the project readme for validated information" and told him it was grand.
His OKR for IC contributions on top of his management responsibilities won't fill itself.
That's why Usenet is full of posts reading "RTFM."
For some reason, instead of telling people to do that, which solves the problem, we just stopped writing manuals altogether.
Did it solve the problem? If so, why people had to keep repeating it everywhere the entire time?
In my experience the text for the Claude has only one requirement - the intent and meaning must be there.
The text for Claude doesn't need structure. Doesn't need style. Doesn't need formatting. Doesn't need deeper thought. The only important thing is that it includes somewhere somehow the relevant bits of information.
The quality of prose I throw at him is below what I would show to any other human. I just turn on my microphone, keep dictating whatever comes to my mind and I think might be relevant. After this is done I may or may not ask Claude to rephrase what I wrote before keeping it as memory.
On the other hand people judge you for what and HOW you type. They complain about it.
It's in my experience that people will generally judge a programmer much more for the quality of his outputs than the number of them. So if your target are other humans - it's better to have no docs than bad docs. For claude it's the other way around.
Now I'm wondering if I'm just making the documentation worse for the one coding buddy I've got that reads it.
If you are ever on a project where somebody goes "Somebody should write some documentation for X", you should counter it with "Great idea, get on it!". Mostly nothing will happen. It's rather thankless work. Some people are more proactive on this.
I actually tend to write documentation for myself. Because I'm old and wise enough to realize that if I come back to a project in two years, I will have forgotten most that I would need to get back up to speed quickly.
With agentic coding tools, it's different. The documentation helps. And it gets added to even if you don't ask for it. Which is nice. And you can get a lot of documentation added with a few simple prompts. Which makes it cheap and easy to generate.
(aka "autodocs", for those who remember the term)
I had an online art class right before Stable Diffusion came out. After SD workflow got well known among the art community, I asked the teacher what's the difference between AI image-gen and human artists. His answer (paraphrased): It's easier to make AI learn.
When you're paying piecework style, suddenly the work needed to write a good spec has a bottom-line payoff.
In a different project, I instead have it maintain a project overview and a couple other docs, and we delete plan.md once the work is done and the docs updated. I like that better.
Claude is a better reader. I have to just tell it to read the docs/specs sometimes.
I used to write extensive docs, now I solely write docs (I mean, the typical automated model Zoo do it for me) so AI know what to do later. Even inside the team, we don't really explain (except very high level concepts) anything for onboarding because the onboarding is directly on the harness, the file structure of a repo isn't even really checked anymore (probably no point, soon enough) as most people will end-up entirely on a chatbot anyway, it's start to be hard for me to even justify going out of our own internal harness windows (I have 16 to 32 of them open with 8 monitors).
I genuinely think a massive wave of depression will hit "tech workers" when they might realize that all our greatness (programming, arguing, planning...) will just be to prompt all day long and we will just be all in a "chatbot" in the end.
The bot though- I don't want it to waste a bunch of tokens exploring nonsense. If I can write a few lines of text that will help it get straight to where I need it to go for 99% of work, that's a win.
It feels hand-holdy when done for the bot. I don't want to hold my follow human's hands.
Similar for adding static type checking, which makes it easier for AI and coworkers to understand the code, catch mistakes, and to refactor. And now there's coders willing to add static type checking to help AI but didn't see the benefit before for some reason.
it's never been easier to catch up to the team guru
I am not convinced that just adding llm summaries to a commit will have long term value, especially if you don’t keep the ‘why’ separated from what is probably going to be a verbose how.
But I would be happy to be shown wrong here.
I think this might lead to more literate programming. The main challenge with LLMs is humans understanding the code, which lp helps with. Also, it includes the relevant context with the code itself. Both of these things help humans and LLMs.
I've been trying it myself and I think it's working pretty well. The only challenge right now is that it is difficult to get models to output code literate style. The output from LLMs tends to open a code block and put everything in it with a ton of long comments, rather than create several blocks with prose in between. [A caveat is that I don't have access to SOTA models.] My plan is to add an agent that just focuses on the style.
I don’t believe spec driven development is a good idea though. The architecture should be made with intention as well, or I feel we‘d end up with whatever happens to be ranked highest in the latent space. But the specs are great to align cross platform teams behind shared concepts, and they are a good input to automatic reviews.
Writing and reading docs used to be incredibly time consuming, and programmers often didn’t read them. Now you can all but guarantee a doc will be found and read and followed if it’s relevant. The ROI on docs is high now.
Documenting for Claude: you put in the work to get what you want.
Seems pretty straightforward.
Unfortunately, companies often measure developers by their own PRs.
An unfortunate outcome of this is that writing docs for other developers isn’t really incentivized properly (and with stack ranking, you might even say it’s disincentivized). Writing docs for Claude, however…
Are you really more productive if instead of coding you are spending your time tweaking the AI to do what you want?
Well of course people adjust their harnesses, skills and whatnot - that is how programming looks like now. You don’t touch the code, you build a machine that builds the code instead.
The question is how much people can produce this way. Me, personally, a ton - right now I feel I can do in a week what would take me a month-two. And I’ve had 20 years of experience in programming.
I ran /insights on Claude Code and it said code review was my most requested activity. I turned it into a skill that autodiscovers the project's structure and launches a huge matrix of parallel critic agents, each focusing on one specific area of the codebase and a quality like correctness, maintainability, security, etc. Supports file system style journaling to deal with subscription usage limits and interruptions.
Took maybe a few hours to fine tune this and then I applied it to all of my projects, and it's actually absurd how productive it is. This is basically an infinite GitHub issue generator. I run it and then start checking off the items in order to definitely improve the status quo. Review again, fix again. Just loop this until zero issues found. The only question is whether to fix the issues myself or tell Claude to do it. I still do it myself in the projects I really care about.
It takes a few tries before they internalize that they need to have a doc link before expecting my help. Once they do, I might take the next step to saying "here's the answer; can you update the docs so the next person doesn't have to ask?" And it might take a few tries before that sticks too. My goal is to eventually turn them into someone who evangelizes the docs themselves.
When I write peer reviews for my colleagues, I describe their attitude toward documentation. If that's "they refuse to open the docs, frequently wasting their colleagues' time", it's not gonna go well. If it's "they make nice doc edits after I ask them", a little better. If it's "they proactively maintain the documentation", better still.
Of course all this is for stuff that one could reasonably expect to be documented. Help thinking through a design problem or debugging their in-progress PR is generally a different situation.
People rag on StackOverflow for being mean, but it was a good training ground for developing habits that satisfy the social contract of professional spaces.
Now, a really good human collaborator who reads all the stuff and thinks carefully, that was still better than what I saw from AI models at the start of this year. But I've also worked with my share of idiots, and been one too.
I'm not going to get into if *current* models can or can't reliably do any particular thing to any particular standard; previously my comparison was the same conversations with regard to video game computer graphics in the 90s always being "photorealistic" when they really weren't*; now, I'm starting to feel such discussions have the same vibes as Tesla fans insisting that "FSD-{insert current version here} solves all the problems and is a real breakthrough and the Rototaxi will totes conquer the marketplace this time for real bro, just one more version bro", etc.
or, in other words , if you want the thing to always read the documentation then make that a strongly highlighted point both in pre-prompts, active prompts, and memory.