I just don't see how you can truly reason about a system without delving into code. Tests aren't enough, running the software isn't enough, high level system architecture isn't enough.

>As is so often the case with coding agents, having experience as a tech lead or engineering manager really helps here ... You need to be able to make smart, informed decisions about that system

In my experience, engineering managers are too detached from the system to accurately reason about the system. Tech leads on the other hand usually can given enough time, but they tend to defer judgement to senior ICs on the team who are more familiar with the code.

Point is: there's no way to have your cake and eat it to. You either read the fucking code and keep a mental model of how the system works in your head, or you have an overstated confidence in your ability to reason about the system (and this has been a problem well before LLMs).

The code change itself doesn't specifically matter. But suffice to say, it was about an AI feature in one of our products.

The code was stamped by Claude driven by a prompt. The prompt was for a ticket generated with the Atlassian AI integration. Atlassian had digested docs made with AI. The docs came from strategy memos I'm 90% sure were written entirely by Claude.

The strategy was chosen by management at the urging of exec leadership. The execs now communicate mostly via AI written memos. I do not know how they make decisions, but they reference tech influencers, market conditions, customer expectations.

This gave me pause. Who had actually made the decision then? Arguably there has been several layers of human review, but the actual source of the decision was hard to pin down.

We were not building the feature because we wanted it. We were building it because we thought other people expected it.

Perhaps reflecting on the state of the market, I thought, could indicate who was actually in control.

Where do investor and customer expectations come from in 2026? It is very murky, at least in tech. There appears to be hype. Some hype comes from true believers, some comes from cynics. But both respond to market incentives that reward bigger and bigger claims.

Where does the market's "action" come from? What is the driver?

Investors do not really seem to understand what the tech is or its limitations. Some are passive operators. Others are just responding to the overall froth and speculation in the market - which becomes a runaway feedback cycle.

This left me lost.

Nobody in this ecosystem, I thought, is actually in control here.

Nobody is actually orienting work and action to real, concrete goals. It's all based on speculation and anxiety about the future.

So it is not only that nobody understands what the code does. It is that we cannot, or at least I cannot, explain the motivation. There doesn't seem to "be" any form of "intention" in this environment.

It has all been hollowed out, replaced either be inscrutable machines, or inscrutable incentives.

Ironically it rather resembles the kind of "misaligned" superintelligence we are supposed to be avoiding.

It boggles me we completely forgot that the world operated like this just 4 years ago

And having AI code to review is no different than any other code that ever was to review, so the review tooling is - as it necessitates - also benefited by lugubrious application of .. more AI. But: all AI is human reviewed.

So it's not a big impact. We just don't ship code that isn't 100% human reviewed, If that's insurmountable: you're doing it wrong. Use AI to make code readable again.

And then, also, put AI back in its box. Don't give devs 100% full-time API access to subscriptions: give them actual hardware to use, to go 100% local.

Local AI is, thus, the best AI, folks. Don't use more than you can run locally, is a great way to keep AI code properly maintainable.

The industry will prove this, itself, sooner or later: If you can't put your AI in its box for safe-keeping, you're doing it wrong, anyway... and should've already learned this practice as a habit, decades ago, vis a vis future-proof tooling... (See also: not logging everything you do with an AI? Big fail.)

Sure, the absolutely intoxicating addiction of Big Metal AI™ is going to put a lot of consumers in a deep, deep pit of Neo-Illiteracy - however: 'good' AI code is actually just good code.

Edit: Since I seem to have touched a nerve - I've been working on a project to solve this: https://www.archme.io if you want to know my thoughts on the right abstraction

If you can really get a good set of requirements, go and write all of your test cases out, and then throw it at an AI that will one shot it. Its perfect.

Here's an anecdote about a way to do this wrong.

I have found with AI coding methods that there's a line where it becomes a hail-mary (in the American Football sense).

A hail-mary is when you throw the ball to the end zone and just pray someone catches it. This is almost always at the end of the game.

This moment with AI code is indicative that you can't put together a coherent plan so you just tell the agent to "make it good". It used to be that the results here would suck, but now the agents are really competent, so the results might be good.

But at that moment, that's your cue to back up. Because as soon as you take a solution that's so far detached from your understanding, you're underwater. The hail mary is not part of a larger game plan. It's the last play of the game. There's nothing after.

So as soon as you reach that moment in your coding, you're signaling that you're done understanding not just the code, but even the way it works at a high level. If you're still going to work with this code after, then back up and work with the AI to get more understanding of the problem.

Aside: this reminds me of running a “negative split” in a long distance race, where you aim to run the second half faster than the first. It’s very hard to do this because you have to be willing to let everyone else in your pace group pull waaay ahead, and running above race pace in the beginning feels “free” with all of the adrenaline. But if you do manage to stay disciplined, it’s a fantastic feeling to reach half-way with gas in the tank, and then start to reel in all those runners who sped by in the beginning.

You can still know things and get force multiplication out of LLMs, if you are disciplined and caring enough. In practice, most people won't be. And you can't force other people to be. But you can force yourself to be.

Do you need all these layers of abstraction when the human is no longer looking at the code?

Best analogy is forgetting how to use a slide rule following the advent of calculators. The former was made to make hand-calculation of logarithms easy. The latter does these calculations directly (obviating the need for a slide rule at all).

I think what humans still need to learn are the theory and domain fundamentals for their industry. If that industry is computer science, that means algorithms, calculus, linear algebra, etc. I think the future of CS is then (a) theoretical human-drive design and (b) prompt engineering to implement and verify that design.

It would also be helpful to have domain knowledge outside of CS as having the skills to build something is nearly commoditized (outside of the above fundamentals).

Scheduled a quick call to align me on what he expects - normally he wouldn't do that but he has attached a big agenda written by Claude what the presentation could show, and invited two other product colleagues of mine.

I came to a Miro board of the Claude Agenda, put into Miro using the MCP.

Honestly, just tiring. Asked my colleagues if they would just put the Claude agenda into Miro with Claude, what they need me for when an AI could just narrate it.

Dealing with legacy messes I used to get frustrated and bored of making the improvements, now it's easy to clean up code bases and write loads of tests. I asked Astra to come up with a plan for Playwright testing the whole App, I have not built it yet but the flows suggested were fantastic as was the ephemeral database we plan to create for CI.

I built my friends portfolio website almost entirely vibe coded in 4 hours and it looks unbelievable, we added so much slickness (he's a designer) just prompting together. I used a CMS I had never used once before and it was so so easy to do without any of the usual need to read docs about everything.

I've done so much devops now I'm actually fairly confident that me and an AI can do anything you want in terms of deployment/infra and scaling from AWS to Terraform to whatever.

Anyway my main concern about this technology is not that it is crap at coding it's that the improvements in how it codes and thinks are absolutely dramatic which is extremely scary - it has come so far in a year I wonder what the next year will bring.

We have both the tools and the skills.

Later, we will loose the skills because of AI and the depletion of natural resources will lead to the scarcity of the tools.

Even with unlimited spend, it seems immensely beneficial to dig into the code base and fix a certain amount of bugs oneself. Oftentimes this is ends up being quicker than having to type out a detailed explanation of an issue in plain language, with the added benefit of maintaining intimate knowledge of the code base.

This is also freeing up time to explore things that before you wouldn't have been able to even start. New fields in tech are opening up. It's all about the model, compute, plugins, third parties... and more to come!

And this is not limited to code, but also how the world works. It's all being abstracted into prompts. Funnily enough, I'm also learning that way...just taking less time to get to the point. But this is not the first time we go through this. Eg. Google vs a library. And like anything, if no one knows anything anymore how do we distinguish from one another? There's a level of wanting to understand in order to distinguish ourselves from the rest in the serendipity of everyday life.

Call me naive, but something tells me we're going to start being much more open to just exploring the world with all this time we just bought ourselves thanks to technology. We were always gonna get to this point and there'll undeniably be bumps ahead.

But after 2 months it starts to backfire me. I still know nothing. I have some understanding of the system design and core components but I have zero clue about how certain things are done under the hood. Because AI read code for me and code for me and I take it as my own understanding.

In last week I end up limiting my AI usage and forcing myself (it is really hard) to read and code at least a bit by myself to start having any idea about what is going on here.

We all interact with systems through mental models, but if many devs are just prompting claude when something doesn't work, they might read what claude found, but lose out on the exploration, debugging, and work that builds and reinforces the correct mental model and discourages the wrong one. And if devs are missing out on the mental models, will they actually be capable of driving efficient solutions to problems as the mental models get worse.

Enough, already.

Does anyone single person understand what’s happening when compiling a large C++ code base? Meaning, can anyone track the basket cast C++ language constructs down to the Clang IR to the optimized machine instructions? From there can any one person follow those machine instructions all the way through to the actual registers etc to actually running the code?

Unfortunately I learned that not everybody thinks this way. Some orgs do imperfectly fine without good engineering discipline, and that has been the case before AI... AI has only made it easier to give the appearance of good engineering, which is exactly the pre-AI goal of many orgs

And I don't think it was ever necessary to go to the point where people just gave up authorship. These were choices made by adopting the "I'll do everything for you" agentic "harness" model that shipped with Claude Code but it was never inevitable.

e.g. we completely dropped fill-in-the-middle completion OG CoPilot auto completion model. That combined AI authorship with a human always in the mix and I actually really enjoyed it. It's just that the models involved were pretty stupid. We totally could have had IDE / shell / tooling integration that kept people in the driver's seat while automating parts of the drudgery away. Instead what we got was a simple chat loop with "oh, whatever, you go do it" being the ultimate result. Cuz, you'll totally review everything after and understand it, right?

The things should end by quizzing you on what was just made and if you don't pass, just throw it away. That'd be funny to watch.

I'd think that's what they call paradigm shift, and this probably repeated across generations from the introduction of the printing press, PC, the wheel, the internet to stochastic parrots that reduced what we still stubbornly insist require our special neurons to mere statistical modelling that can be aggressively scaled.

The issue of course is that if you do invest the time, then you're no longer saving time by using AI. You're just spending it reading and trying to understand something you didn't write. And that can be unpleasant in its own way.

My hot take is that for parts of a system that can be considered its core, forming a deep understanding is almost always important, and so is knowing how the different business domains integrate and where the connection points are. For many others, a high level understanding is sufficient. The difference is that now, with AI, you can make that choice. Before, you had to write everything yourself, and for any sufficiently complex and long-lived system it became impossible to hold all of it in your head.

> And having AI code to review is no different than any other code that ever was to review

If your company is sticking to "everything needs (human) review" then you wont run into most of this. This issue is that a lot of companies are using AI as an excuse to remove that review process (either partially or entirely).

I have strong disagreement because it sounds like, by analogy or proxy, we have also "solved writing"

Except a little worse, since they were raised alone in a library, act mostly the same, and have harsh limits on personal growth.

My team recently spent two weeks on a wild goose chase trying to figure out why TensorFlow Lite was generating nonsensical OpenCL kernels. Well it turns out that LLVM had a few bugs in the RISC-V assembly for our platform that was leading to silent garbage. It took combing through assembly dumps, hexdumps, a lot of pain staking debugging, and going through the TensorFlow Lite source code to to track this down.

In your opinion, if code is the wrong abstraction to be working at, how do you approach this scenario?

Which is really the same problem with coding.

The agentic model of it just taking over and doing everything is poisonous to effective long term team work.

We're well past the point where it's about the quality of the work they produce. It's the way they integrate (or rather, don't) into human practices.

While these criticisms were technically true, they were stated mostly out of a sense of insecurity from people whose jobs were basically spending years and years just glueing code together and mixing APIs to display some CRUD apps rather than out of a genuine concern for whether LLMs were actually producing poor products.

Nowadays those concrete criticisms don't really work anymore, LLMs are pretty good now and surpass most developers when it comes to writing the majority of shovelware that people have been employed, and so the narrative is changing from concrete criticisms about how LLMs were genuinely not capable of writing software... to these kinds of abstract and philosophical arguments that are really hard to argue against because they make no concrete claims.

If you say an LLM can't implement a feature, we'll we can test that claim concretely and LLMs are getting much better with every new release. If you say its code is slower, buggier, or less maintainable, those claims too can be measured and once again they're getting really good at these. If you say it takes longer to complete a task or requires more human intervention, we can compare it and measure etc...

But now the objection has shifted not to LLMs are incapable, but people are now incapable and LLMs represent a degradation of the "craft". And here there is nothing left to test or falsify. The argument has stopped being about whether the LLMs work, because that's verifiable and they are now at a point where it's hard to argue against their ability to actually produce functioning software, so now the argument is about whether people are morally, culturally, or intellectually permitted to use it.

Whatever the path is, regular engineers (90% of the people around here) will get screwed up one way or another. But hey, playing with LLM agents is cool!