We automated everything except knowing what's going on(eversole.dev) |
We automated everything except knowing what's going on(eversole.dev) |
Perfect summary.
It's like we invented a world where you can finally, _finally_ speedrun an enormous legacy codebase and all patted ourselves on the back like that was a good thing.
We have reached a point of complexity and short-termism where it's standard practice to shove a huge, barely tested mass of Python, JavaScript, shell scripts, and who knows what else inside a docker container and call it done. Complete with hundreds or thousands of intractable dependencies, many of which we know ~nothing about and thousands of lines of esoteric configurations for servers we have barely any hope of even getting to run optimally, let alone securely.
Most software has been awful for a while.
Already, with AI:
- We can build everything in a statically typed, analysable, and memory safe language like Rust[0], even bits that really have to interact with the shell or wider OS and would previously have been shell scripts
- We can insist on positively deranged amounts of testing, at every level
- We can easily cut the number of dependencies in our code by >50%, in many cases more like 90%
- We can do the refactor as soon as it becomes obvious that it would be a good idea
- We can implement quality of life admin and monitoring features that would otherwise sit on the backlog for eternity
- We can educate ourselves about what we've built[1] by asking questions about our codebase, build tools to understand the behaviour of our systems, etc.
So yes, I agree that "The Future Belongs to Whoever Understands What They Shipped", but unlike the author I am somewhat optimistic[2]. There is more opportunity than ever to build and understand extremely high quality software that does not accept technical debt, corner cutting to meet deadlines, or poor quality (in design or implementation), for those that engineers who are knowledgeable enough and willing to embrace the new tools.
And AI, and the tooling around it, is only getting better.
[0] or Go or even TypeScript, but there's precious little reason not to pick Rust for most use cases now
[1] of course we need to choose to, and many won't…
[2] of course, there'll also be near-infinite valueless slop and some people will get sucked into that, but this seems little different to regurgitated SEO spam, short form video, and all the non-AI enshittification we already put up with, and perhaps AI will help more of us do a better job of avoiding it
But when I'm using all of those things (pickaxe, mineral mine, power station, internet network hub), I know that there was a thinking human being that took some measure of human care and consideration when creating them. And that there are people on the other side of the economic transaction to talk to or hold accountable when something goes wrong.
If we could conjure pickaxes and electric power plants in a single day, that would be speedrunning.
AI is very useful, but it so far doesn't write the type of code I can trust. Thus I use it but I carefully review everything it does.
He told it to calm down and just use php, it gave him 100 lines with no dependencies that worked the first time.
The Pieter Levels stack :)
Of course, this is ideal for solo entrepreneur. If you are employed, then you cannot finish it in 100 lines. How will you get paid to maintain it for the next ten years, and hire all your friends to help you?
I think this difference in incentives explains most of what we've been complaining about for the last twenty years.
But the point remains: the NPM monoculture is indefensible.
Very few understand deeply what's happening within the computer between the cou and the bridges and the rest.
The fdiv bug in 1994 took us all by surprise because we were in a situation where bug couldn't exist in hardware, because it either works or it doesn't.
When I'm using firebase or aws, I don't know the underlying system, I don't know why some resources can be created with an underscore or other can't start with a number.
Yet it works.
We're working in layers where usually we only touch the last one. Yes, understanding the others is great to debug.
I'm even wondering whether we need tests when they are written by the same llm that wrote the code.
Of course we do. Otherwise we start trying to water crops with Brawndo
> Very few understand deeply what's happening within the computer between the cou and the bridges and the rest
But it's very very important that those people deeply understand it. We cannot replace their actual knowledge with LLM approximations of their knowledge
But it got electrolytes !
For now it seems we're veering towards even more specialization in roles.
The concept of "Full Stack" might disappear because, well any LLM can center a div now and create the sql query that fills it with data.
When asking a LLM about "any sre interview question that LLM got wrong ?", it identifies the questions and the corrects answers.
I should take some time to discuss with an LLM to see their answer for one of my last assignment where the underlying issue revealed by bad performance was simply that the client shouldn't have used relational db for almost immutable documents.
And that they should have bought a math library instead of writing their own.
But I don't see it. Where is this glut of software?
I suppose if one is simultaneously ignorant when it comes to software and an expert at agentic workflow, then yeah sure, maybe--at the cost of how many tokens, though? But logically it seems that the former would preclude the latter.
Also, the "get it done in a weekend" seems to be a gross exaggeration.
- CLI voice changer with cloned Overwatch voices on ElevenLabs.
- Brother P-Touch label maker using HTML/CSS. Their app is absolutely atrocious.
- Converted a FileMaker CRM into a Next.js/Supabase app.
- Dozens of drag-n-drop or 1-click/CLI tools. Think flattening a folder, a zip file.
- Dozens of Chrome Extensions and TamperMonkey user scripts. Think blocking ads with very targeted xpath.
But when I think about sharing them it feels like what's the point since anyone can make them themselves?
Not saying that's how it should be, that's just the world I predict in the not too distant future.
I love thinking, but most people I know seem to experience it as a form of physical pain.
by managing the risk of failure and technical debt with a financial instrument, i have a lot more freedom to move fast and break things, and scale aggressively
but if you just buy deep out of the money put options every year, say 50% price drop, in a 10b5-1 structure, its ok. you will get sued a bit more than usual
not much different than buying life insurance, except if your company crashes like monday.com , some schmuck who sold you the puts will have to pay you a tonne of cash then you can do a dilutionary rights issue or just use the cash buy a boat in miami and start something else
Maybe you're looking for 'moral hazard'?
Others are pointing out that you cannot understand everything - and that's true enough.
But you only need to understand what's important. The experience of a good expert helps you to find that out.
As a systems administrator the recent AWS outage in the Middle East is the best recent example. There will be roughly three types of companies, separated by their understanding:
- Don't Understand - these companies thought that the cloud would handle this kind of thing for them, and are probably going to be doing a lot of finger-pointing in the near future.
- Do Understand, Don't Care - these companies did understand that high availability meant going multi-region, but decided against it for whatever reason. Probably cost vs perceived likelihood. These companies know that they've made a mistake. Short term they're wondering how to survive it, long term they'll be re-assessing their risk acceptance. Many may decide to stay single-region, but at least understand why.
- Do Understand, Do Care - these companies will simply be checking that their procedures worked for any manual parts of their failover, plus possibly looking at any improvements they can make given the real-life experience they've gained.
An LLM is just going to tell you how to implement it. It's not going to be thinking "what sort of availability do we require?", it may never start that conversation unless explicitly prompted. And even then it's going to return consensus opinions, which may not be what you want when evaluating risk.
I'd love to think a lot of companies will be looking at this event and updating their own risk register or justifying their existing risk decisions for hosting. But let's be honest - most won't even have thought about it, and won't until it goes wrong.
No one understanding what's going on inside of complex systems in financially constrained environments built and maintained by average, at best, engineers is the norm and is what keeps the world running.
None of that is a symptom of AI. The only change AI brings is that even first person developers don't know anymore what the fuck they just deployed.
I wanted to touch on this point but then this post started getting WAY too long.
>the only change AI brings is that even first person developers don't know anymore what the fuck they just deployed.
This to me is going to make your first point 100x worse in every damn way
You need to run those systems in parallel, integrate them in each other or have an abstraction/two-rails layer in front of them, test, test, test and test again, all of which takes months, while new features now need to be added to both systems, whether by AI or some average developer schmuck!
Do you have any idea how much any of this costs? Mind boggling amounts of money. Infrastructure costs will at least DOUBLE for several MONTHS. Meanwhile you will need extra manpower and AI credits for MONTHS on top of your increased existing costs for maintenance and new features.
Welcome to a new hell! LLM can't solve this!
ASM was machine specific. C was portable but required expert programmers. C++ was even more user friendly, but still very hard for normal people. Today, most anyone can write a program in Go.
The more we abstract, the less knowledge/expertise is needed. So yes, programs are being built by people who don't really understand what they are doing. That is intended.
Most of you won't be able to answer that. And you already know it.
That's the conversation this industry needs to have. Not tomorrow. Now.
There are reports of industries trying to use these tools to generate as much as possible, sure. There are also people generating bad art and unpleasant prose and using llms to generate nonsense they don’t pay attention to.
I don’t see why that implies that you or I lost interest in tinkering with toys we build. If I want to spend 4 weeks understanding oauth a little better by implementing a client, I still can and I still do.
Automating our builds absolutely didn’t create a cathedral of complexity while nobody noticed. It did mean I can open an Free Software project, read the build file and understand how to build the thing. That’s the opposite of generating complexity.
I worry about our future generations as much as the next person, but this low effort pabulum doesn’t represent the thoughtful industry and hobby that I love.
For most businesses, software is just a means to an end, they don't really care how high quality and thoughtful the systems they use are (e.g. look at any piece of "enterprise" software)
What LLMs have done is made much much easier for orgs to launch new features and services both internally and externally, without necessarily understanding the complexity.
For me, that's what this post tapped into. Many orgs already have more complexity than they can reasonably handle. Massively accelerating development, is not going to make that problem better :)
Don't get me wrong. I think the future is very bright for software. I have friends who are scientist and biomedical professionals and I am excited to see what they are able to do with the powers of software where they don't need to care about syntax and can only lead with their intentions.
The rage bait part is mainly my frustrations manifesting. As an SRE my annoyances came off a little bit when it comes to how fast developers are shipping vs how fast our rails can maintain things.
No. Software is being centralized. If the snake oil AI companies are selling about the coming agentic age were true, then the end result is not "anyone can produce software" it is "anyone stupid enough to rent the ability to run their business from an AI vendor can produce software".
I 100% agree with this in a individual person sense, but in a humanity sense someone does understand linux very deeply and is very intentional on how they change it which to me is how I gain trust in it.
does trust change when the entire SLDC is AI?
There are systems still in use from the 1960s (maybe before) - the original authors are at least retired and likely dead. I question how well the replacements understand all that. Sure they have had to dig in and understand some parts, but what about the parts that just keep working and don't need new features?
If your answer is "yes," do you think that's inherent to the (metaphysical?) fact of it being AI or to specific limitations to current AI? If the latter, what changes to AI would let you trust it?
Maybe they stick around for a while, maybe they move on to another job, but they were THERE at some point. They have a name. You can ask them questions about what they did. And hey, they still exist in the real world too, so you can get in touch with them even after they leave if you need to.
AI powered development is like a guy shows up, gets hired for 90 seconds, writes part of a feature, and dies instantly once the code hits the screen.
Over the last couple of months I've seen a load of new "product launches" in my niche but when you look at them they're largely vibecoded and don't show deep understanding and sustainability, so it's pretty likely you'll never see them as successful businesses.
Looking at some of the related places like /r/sideproject/ there's a lot of releases and I'd be willing to suggest that most of them are using LLMs
There seems to be a lot of hype, and has been for years, but I’m not seeing it materialize as actual economic output. Surely by now there should be lots of businesses springing up to capture all of this value created by vibecoded software.
Whether that translates into more value for those larger corps is the trillion dollar question :) Writing code is a small part of the process of finding and shipping features that customers want, so it remains to be seen how much LLM tools translate it.
I think it's fairly widely accepted that from a financial standpoint we're in an AI/LLM bubble. There has been more investment than we're likely to see financial benefits, but it's impossible to predict to what degree (if you can predict that and the timing you can make a lot of money!!)
The question is does AI understand well enough to maintain that thing for whatever maintenance I need to do in the future?
But it also kind of sounds like you're just saying that there's a scalar of "get a little better" before you'd trust AI with something.