Coding expertise is going to collapse from AI reliance(larsfaye.com) |
Coding expertise is going to collapse from AI reliance(larsfaye.com) |
Very interesting is how divisive is this statement. In that almost every response is either hard agree or hard disagree.
Does agreement come from observer judgement, or lack thereof?
Compilers take an agreed upon specific intention “implement X via Y code in order to achieve Z” and make it executable. LLMs in the hyped methodology just take “Z”, and decide on an X and Y.
They’re doing things a compiler doesn’t: picking an approach/behavior, and picking a way to perform that approach in code. Those are subjective judgment calls, not mechanical processes that could be encoded in a classical program. That saves time, but given their reward system (“what the user requests is always the desirable goal”), they add considerable risks to the process of software development as well. No amount of guardrails and code quality agents can sidestep the fact that the decisions they make are substantially unpredictable and only as good as the prompter knows to ask for.
When that happens, (developer thinks using an ORM means they don't need to know how to use a database,) the project eventually fails. It's the same thing with AI; just like companies learned that they need to make sure developers actually know how to use a database, companies will learn that their engineers actually need to know how to program.
Maybe it's time we have a new or more universal coding language we teach in schools that is strong on concepts and standardisation. Something that enforces the math, logic and mechanisms that underpin modern software. It could be designed so that any common liguisric language could apply (as in English, Japanese, French, whatever) for the broadest appeal.
LLM could then translate architecture and concepts into the specific actual coding language of choice for the best fit. Of course you could also read that code and write it yourself.
The whole process could be redesigned with this concept being sort of like a "super compiler". Or hell - if you really wanted to there could be dedicated ASICs with the supercompiler LLM burned in (like Talaas) that could do this on the fly or very quickly for different applications.
- Writing (starting from Platos Phaedrus)
- Printed books
- Photography
- Typewriters, keyboards
- Telephones
- Recorded music
- Television
- Wikipedia
- GPS navigation
- Smartphones
- Automation in aviation, medicine, industries
Of course, he's just one man (or two, if you consider he was conveying Socrates' thoughts, as well) who lived in ancient times and couldn't possibly foresee how technology would evolve...but when I went back and read his concerns, it was ironic to admit that much of what he warned about still applies to this day. People are far less informed despite having access to more information than we've ever had because we've exchanged the ability to remember for the ability to just look things up. And, we also have less meaningful dialog than ever before; people just sit on social media, copying and pasting "facts" to each other, instead of having actual productive discourse.
AI tooling can be a boon to learning, but it requires us to stop using them for code generation as a primary purpose (at least for juniors) and instead advocate for Socratic workflows that still require manual coding practices. When someone decides to code something, it's not just learning syntax; it engages a variety of mental disciplines from critical thinking to planning to creative problem solving to logic and math, even.
I mean, they were right, then, too. My meaning is that expertise already collapsed several decades ago.
--
One of my favorite talks of all time:
Jonathan Blow, Preventing the Collapse of Civilization (2019)
It is important to ensure that any LLM code is named and shamed.
You need to take it further.
We need to shame and mock the people (read: delusional imbeciles) who think this is acceptable if we want it to stop.
AI users need to fear for their careers and reputations if we want to get software development back on the right track.
- it is going to be so bad that vast majority of these "i dont write code anymore" guys are going to be unable to write even a c++ hello world program without an LLM
- the argument these guys come up with all the time is "I dont need to"
- Big assumption there buddy, big assumption. Lets play both cases shall we.
- Case 1: LLMs infinitely improve and nobody has to code anymore. Yea well, writing a paragraph spec isnt that hard for me bro, I already do it for every project while not using an LLM
- Case 2: LLMS go bust completely for whatever reason. We have a whole generation of mass programmers and 99% of them cant add 2 numbers in c++. Guess what? I am now one of the most sought after programmers in the entire world and part of an absolute minority
It kinda seems like wishful thinking on our part as software engineers that one day we'll be able to go "Hah, miss me now?" but that just isn't the writing that I see on the wall.
Over the past year, I've AI-generated two large pieces of software over 2000 commits without having much depth in either domain, and my ability to deliver value as a veteran software engineer has only shrunk over the months or it can be trivially extracted into a reusable markdown file like "design bar: ensure things 'by construction' where possible", a lesson I learned viscerally over decades.
I don't mean to pick on you, but to present this in the service of this kind of fatalism... you know the inevitable follow-up question, right?
Because I can still reason and dig into things and ask questions and have the domain explained to me out of curiosity, the same way I could build anything that was originally out of my depth.
But what I've noticed is that the need for any corrective power has gone or is going to nil and I'm mainly doing directional work. And even then, I can constantly have sota models "rank the top options and recommend one", and it's almost always the right way forward.
I used to read every line of every plan that LLMs generated, but over time I've realized I have less and less to correct or, more impressively, the LLM had foresights I never considered. The same thing happens with implementation.
Periodically I can spawn a bunch of agents to evaluate the system adversarially to find improvements, and the findings have been so good that it's evident that soon I can just automate that too.
In other words, the skills I need to excel here are more curiosity and patience than tech expertise -- the things that got me into software engineering in the first place since that was the only way to build things.
- Photography hasn't erased other visual arts. Both require a separate skillset; both have their uses and their enthusiasts. Photography benefits from having a lower skill floor, in my experience doing it, and this in turn has led to commercial photography eclipsing commercial painting (barring things like painting buildings) as a viable business model, largely.
I'm not sure if we're better or worse off as a society for having gone through that shift.
This has more- and less-pronounced areas. In the US (which is geographically close to me and which has an outsize proportion on my own country's culture), Fox News is an excellent example of how TV can erode critical thinking.
Just like with TV, smartphones have clear benefits and utility in daily life. They aren't abjectly bad inventions, but they have a serious societal toll, and it's difficult to ignore that.
In the wake of that accident, many aviation regulators instituted or began requiring that manufacturers and airlines formalize training and operational procedures that protect thoroughly against this sort of cognitive gap.
In software, we like to think that we're a serious industry, sometimes. We spent much of the 2010s going in what I found to be a good direction: the development of IaC, the popularization and standardization of tools like Git, registries for vulnerabilities like the NVD, and even the development of languages like Rust which address historical technical shortfalls of high-performance, low-level languages like C; but we always fell short, industry-wide, on matching these toolchain upgrades with technically-rigorous procedures.
For every borrow checker, there's vibe coding.
First they fuck up your codebase with so much slop that no human does (or could) understand it.
Now if you want to keep making bug fixes, you're on the hook for whatever Anthropic wants to charge you.
The solution is to make AI developers the scapegoat. Fire the people at the top of the AI leaderboards, with malice. Otherwise, lazy AI-addict developers are going to destroy what remains of your company's IP.
https://ivyleaguecenter.org/2024/03/12/over-reliance-on-calc...
Math fosters critical thinking, and we've been in a critical thinking deficit for decades and decades. I don't think we can afford to sink even deeper...
I kinda agree, but I also kinda disagree.
My question would be if "math" is the right tool for that. It can achieve that as a side-effect, but how we express it and talk about it is full of legacy cruft, cargo-culting and general bullshit that makes it not exactly snap into place for all kinds of brain wirings.
Math I believe is not an end in itself. Maybe with that perspective, solutions that satisfy our needs there can be found
Compared to when?
Seems like critical reading skills have already collapsed.
Online discourse though is collapsing under dumb AI talking points.
Now, they all did have gems of interaction that would could find by sifting thru the dumb stuff, but I find that property of discourse to be pretty steady. I've even found some interesting comments on YouTube videos and newpaper websites.
This is true and nobody complains that GCC or LLVM is making programmers dumber. They are code-generation tools from higher-level source code. Much like LLMs are code-generation tools for higher-level language/prompting.
"Oh, that old simple device didn't displace anything, so this one which is now quickly taking over every job will be just like that simple one". So I guess people today know how to sew and knit and bake bread?
"Oh, humans were always needed, so it'll be the same this time." Is that true of horses and oxen after cars and tractors were invented?
Also, yes I am dependent on a calculator.
https://ivyleaguecenter.org/2024/03/12/over-reliance-on-calc...
This is more like paying someone else to read your math homework, generate answers, and maybe even submit them on your behalf. If that had been the (average) student experience, math skills really would have collapsed.
Assuming we can re-establish the will for democracy and taking care of the whole group.
People can no longer drive a stick-shift car, whereas it used to be that a valet would always be able to drive one to park it.
However, I can use mathematica to demonstrate the criteria for various forms topological stability of solutions to many types of differential equations. I'd rather live now.
so many folks struggle to calculate a 20% tip in their head
llms have absolutely made education much more complicated for educators
With a calculator you still need to understand the fundamentals of mathematics. You also need to understand how to operate a calculator and know its functions.
Even when removing numbers and manual calculation, being a function or formula oriented mathematician still requires you to know the theory and functionality.
With an LLM, you don't even need to know anything other than asking the robot to perform. You don't understand the output, nor can an unskilled person explain it.
I know, all irrelevant little philosophical roadbumps. The artifact satisfies the buyer. I think I'm going to have to step away from this industry pretty soon, at least for a while.
A lot of my nonchalance here is from dealing with an immense sense of loss for the end of my identity-level skillset/profession, then acceptance, then deciding to push the envelope on AI-generated software to see what I can pull off since the start of this year.
Focusing on creating things that I want has been an antidote to the melange/uncertainty for the future.
And that was fine! It was a pain. But people need to learn. If you had experienced people around who did know those things then everyone would learn and improve. Most of the time.
Also, we were limited in how much code, and therefore how much slop, we could produce.
Now everyone has a firehose aimed at production. It’s hard to keep up.
You'll still need to learn stuff and learn your craft but if you don't become an expert at programming switch statements so much the better or the next garbage web framework.
If you have a truly hard problem - sure write some code or take a hybrid approach. Still, let's not pretend this is what most people are tasked with.
What is valuable now is productivity. Which in today's day and age is the ability to quickly and efficiently drive AI.
Hence the never-ending growth and spewing out of tooling: harnesses, TUIs, multiplexers, agent frameworks, software "factories", etc, etc.
Software engineers are chasing opportunity and expertise in the only thing that's left.
true, but he missed one point - we already leave in such world. And pretty long time.
He mean ofc buying intelligence of AI. But we already buying intelligence of humans.
1. function trampolines, such as in libraries like ATL (Microsoft's COM template library);
2. hand-optimized encoders/decoders for things like ffmpeg;
3. niche applications such as Ben Eater's 6502 breadboard computer (using a modified WozMon and Microsoft BASIC);
4. reading ASM output from C/C++ code in things like the GodBolt site for performance comparisons/analysis;
5. reverse engineering old DOS games;
6. people working on things like compilers (generating optimized assembly), JIT engines, or other code generation -- though this will typically be machine code, possibly with assembly instruction comments;
and other performance critical places where assembly can speed up that part of the code.
Despite that, some other parts of the business are trying their hands at playing slop slots...
It's impossible to buy the FUD that expertise will somehow vanish because people use AI. People come in a variety of capabilities, challenges, and personal interests. Expertise varies from person to person. Those with personality traits that are more likely to take risks and persist against adversity will be the innovators and experts. We've seen this played out countless times.
You have to ignore everything you know intrinsically but can't articulate about innovation, technology, and evolution to drink this FUD Kool-Aid.
Stop the FUD for profit
The long term impact of AI on society is exclusively a net positive if utilised in the correct way. But most of the westerners lack faith in government fiscal policy and think that they are going to be left destitute when they can no longer provide for themselves. When in fact UBI is going to be the most likely outcome.
They are myopically focusing on the short term labour market effects of AI on their own profession instead of realising the long term utopian societal outcome.
"We need to ban AI because I'm going to lose my job!" - This is only said by someone who is winning in their society and wants to maintain the status quo not by those who slipped through the cracks.
[copy and pasted comment from a similar post]
Hi, I'm Earth, have we met?
That's the neat part: you don't!
Even Robert "Uncle Bob" Martin, who's as AI pilled as they come and doesn't read code any longer, says that junior developers shouldn't even look at AI tooling for the first three years:
Programming languages are (largely) deterministic. LLMs are not. The boundary between an abstract concept and something executable sits comfortably in the mind of the programmer in non-LLM contexts. This boundary shifts when using an LLM and becomes shared between the programmer and the matrix multiplication machine.
The "layer of abstraction" analogy is therefore technically true but essentially false. "Just another" papers over how seismic of a paradigm shift all of this is.
In contrast since then we've had almost not noteable physicists or Computer Scientists that I'm aware, we've got a couple, Hawking and Knuth. But apart from that I don't know if there is a single scientist I could point out today that has revolutionized our understanding or even done any real breakthrough work in the field of Physics since Einsteins era.
You also seem to be restricting yourself to pop science knowledge of who’s made breakthroughs. That’s because our collective understanding of physics has become much more detailed, to the point where it starts to lose the ability to be meaningfully comprehensible to the average layperson.
Consider AdS/CFT, for example: https://en.wikipedia.org/wiki/AdS/CFT_correspondence
There’ve been many major developments just in that area since the early 1970s, to through the late 90s all the way to 2019 (the island formula), with many major contributors like Bekenstein, ‘t Hooft, Susskind, Maldacena, and many less well-known names.
If you resurrected some of the scientists you mentioned, they’d have a lot of work to do to catch up, and making field-changing major breakthroughs would be just as difficult for them as for anyone else alive today.
Something similar is true in CS. There’s been plenty of progress in all fields of CS, but similar to the situation with physics, its nature is not as simple as what the founders of the field were dealing with, so there’s less awareness of it outside the field. Just to pick one small example, Conor McBride’s insight that the derivative of a regular type is the type of its one-hole context ( https://news.ycombinator.com/item?id=34358663 ) is beautiful and kind of amazing, but not a result that you’re going to hear much about outside of PhD programs and the like. CS is chock-full of such work. The entire field of programming language theory has come a long, long way since Turing and Von Neumann’s time, to pick a broader example.
But my reply was I guess implicitly assuming the post I was replying to was talking about the median level of critical thinking. Your reply is talking about the peaks of creative accomplishment which is not what I was addressing.
In other words, a positive outcome with thing X is possible but there are also big behavioral and cybernetic pitfalls, ones which are suspiciously persistent.
Ah, yes, a very BOFH approach [0] to workplace quandaries.
> I mean I use coffee quite a lot
Perhaps, but I don't think either of us would agree with the claim: "The way we integrated caffeine for productivity offers a useful and practical model for integrating meth for productivity."
There's an extremely detectable difference in the feedback loops that form between the tool and the people using it. There's something which remains unsolved ~90 years on. If we assume for the sake of argument that coffee is "good" and "productive", then that leads us to the question:
Are these LLM systems going to be more like coffee, or more like meth?
Eliciting the right requirements and user needs. Creating a sound architecture and design. Validating final product meets the requirements. Ensuring there is good test coverage. You know `Engineering`. Mechanical and Electrical engineers didn't go away because of CAD.
CAD is also useless to Mechanical and Electrical engineers who didn't learn fundamental the skills that it streamlines.
that is a very good point! I'm going to use it in the future.
Besides your first point, the rest can be delegated to AI as well. And you don't need to be a "software engineer" to spec out requirements and user needs.
Agents ideate, architect, build, test, and ship the thing.
No, we do not have that word for that.
Yes, we have that word. No, it does not mean that.
Any LLM proofreading could've told the author that, for that matter.
__
What Fingerspitzengefühl actually means could be described as "tact", a precise approach to something and general attention to detail that leads to success. Or.. the lack of all of that leading to blunt failure.
__
I would of course comment with more Fingerspitzengefühl for the emotional needs of the author if the writing wasn't just a sales funnel for their courses and whatever else (+ posted by them themselves.)
I feel pretty good asserting that, as always, a few smart leaders will recognize early that quality and stability are the new game now that we have speed 'for free.' And then there will be incumbents whose eventual demise will merely be accelerated.
LLM chatbots are good at making you feel good. Facts don't care about your feelings.
E.g. I've gotten pretty good at git thanks to AI because it forced me into more complex workflows. I still can't remember the commands but I really grok the concepts and terminology on a far deeper level than before, where I made a conscious effort to stay on the happy path lest I get into some odd state I don't understand. Pretty sure those "odd states" would seem quite clear to me now.
Why? Because people are going to get so much more ambitious about the stuff they build with AI, that code expertise for the stuff they are building is already scarce anyway. People will choose difficult and less popular languages, and will replace open source packages with stuff built entirely in-house by AI.
Going forward, there is no choice except for AI to fully replace the need for code expertise. Depending on humans won’t be scalable.
I think that would feel like crack for some people who are addicted to using or building stuff with LLMs.
There is something similar happening with AI and most people don't understand it and we don't have enough long term studies for data to make conclusions on this yet. Engineers sounding an alarm here but they also don't have the data to support their worry. Certainly some of the lower level understanding of coding languages aren't being acquired and maintained as much as before. Whether that underlying knowledge actually needs to take up the learning bandwidth that it was taking is the real question. My hunch is that it doesn't and that dropping it makes room for higher level concepts and understanding that are actually more important given modern tooling, the same way we don't actually need to know binary and assembly to be effective engineers, but time will tell.
Genuine curiosity in learning how stuff works, not necessarily for financial gain, e.g., from a salary or using web to deliver ads
No real "pressure" for hobbyist to use "AI"
IMO as end user, some of the best software available was originally written by a single unpaid author, not salaried teams
Some of the worst software ever written was produced by overpaid teams
Hobbyists may use relatively old, handwritten code to learn from
That's not necessarily bad. Software quality has declined over time
1784840004 | AI Coding Will Prevent Expertise | https://larsfaye.com/articles/ai-coding-will-prevent-experti... | https://news.ycombinator.com/item?id=49027909
1784909478 | AI Coding will Prevent Expertise The need for friction in skill formation | https://larsfaye.com/articles/ai-coding-will-prevent-experti... | https://news.ycombinator.com/item?id=49037730
While I have been feeling better for quite some time, I feel another starting to brew. While I was never one of the fabled 10x or rockstar developers, I still derived a lot of enjoyment and and sense of purpose from my work.
With the utility of LLMs becoming more ubiquitous by the day, what purpose can I provide anymore? I struggled greatly to get where I am today, and while it might not be impressive to any of my fellow programmers, I am still proud to be a programmer.
I obtained a degree in CS because I found the entire field to be fascinating. I had great passion for learning many of the subjects. What I learned in my degree changed my life forever. I seriously went from one of the, "What's the point in learning most math if you never use it?" to being absolutely infatuated with the subject. (I'm still bad at math, but that does not detract from my pleasure).
With the utmost certainty, I did not push through all my struggles just to prompt LLMs all day. Maybe's its my insecurities manifesting, but I want people to say, "You did a great job" and not "Claude did a great job." I understand the Buddhist were likely correct. My suffering really does come from my attachment, but at the same time, I do not want to let it go.
I have no opposition to using LLMs as tools. I think a fine balance can be achieved. LLMs can even be exceptional learning tools when used appropriately.
If programming is just a means to an end to you, then you will receive no ill judgements from me. We all have different motives in life. But programming has always meant something more to me.
In fact FEM is arguably more deterministic than the old school pen and paper methods (yield line analysis etc.) which often rely more on correct judgement of the structural engineer doing the analysis.
So I don't think the analogy quite holds
We've been hearing about the demise of skills since before the invention of the loom. The only skills we've truly lost are for things nobody uses or makes anymore. What actually happens is, we develop a new tool that's better than the old tool, and we get good at using the new tool. We then have a class of people whose job it is to maintain the new tools. We don't need that many of them because the whole point is to eliminate that labor.
Your jobs as software engineers are going away, it's that simple. Your new job is to use the AI tools to make products. Somebody else's job will be to make the AI a better software engineer.
Not in companies I worked for, no.
> How many of you know Assembly well? How many of you can decompile a program, or write your own compiler or VM? How many of you could pass a computer science course (that you didn't take at Uni)?
None of these was ever an expectation most coders. Some are bonkers as an expectation for all.
Very few people know how to read assembly, instruction sets, binary or even transistor combos. All those layers have been "left behind" to become specializations.
In many ways coding is always about abstracting and LLMs can be a form of "higher level coding".
There's lots of problems with this argument, but it's what's happening and we'll be okay. We've always been.
Report back how productive debugging is and the velocity of adding new features to a codebase you can't reason about.
Without strong coding fundamental, this is what it will feel like for the future of the profession.
A business won't die just because an LLM is the only thing that can reason about a particular code base. At least, not like if a company's own engineers couldn't reason about their own code 2 years ago.
I recognize you don't agree with it - our world is changing.
This never happened. The people who understood and worked with all of that still exist. I wouldn’t be surprised if there’s not more of them now. What happened is more people who didn’t need to know all of that joined the circus.
For example, I have a vibe coded Python monstrosity which I use for stitching together scans of large format negatives. The enlarging lens that I use for scanning has strong chromatic aberration in the red channel but is otherwise very sharp†. I got Codex to figure out some way of using the green and blue channels to sharpen the luminance while preserving the color information from the red channel. I sort of vaguely understand how this works, but not nearly well enough to review all the horrid numpy code that implements it.
For my use case, there is really no reason to look at the code, as I am only interested in the output. As disconcerting as it is, I feel that this is going to become more and more normalized. Looking at the code that your LLM generates will become like looking at the assembly output of your C compiler.
† This makes a lot of sense, if you think about the color of a darkroom safelight...
The article links to this as the study: https://dl.acm.org/doi/epdf/10.1145/3632620.3671116. It doesn't appear to be related to JetBrains in any way?
https://www.youtube.com/watch?v=HTUh0OO6Kmo
The video seemed to indicate it was a study they ran (she refers to the examples as "our subjects were two of twenty four participants"). It seems they are just citing the study. While it doesn't change the study or the takeaways, I'll update the post for clarification!
But I empathize with the author. I think the frustration comes from the fact that other folks in your organization use these tools to “get by” more effectively then before. In the past, a disinterested engineer might ship less under the umbrage of a difficult engineering problem. With the pressure to “ship more” now that there’s AI, the same engineer is using AI to spam PRs to prove they’re working, even the quality of work is low (because quality of thought is low).
What if there's always been two general buckets (or more)? One bucket are large systems that must perform to spec, and the rest are "good enough" systems to at must simply not blow up enough to be useful (see: almost every enterprise software ever written)
I'm likely wrong. But it's an interesting thought experiment. Humans have produced shit code for a long time, and it's not clear that clueless humans + shit AI code is something the market will reject.
A much easier way to force friction is to expect everyone to read the code they are producing. Yes read all of it and understand what it's doing.
A lofty bar sadly.
I know how to create a multi year P&L projection from scratch. It gives me some insight into how much capital a startup will need to get to breakeven. A good AI tool that creates that same spreadsheet in minutes should also be able to explain why different businesses have different shaped cash flow troughs. I don't think people using tools like that is bad even if they don't gain deep insight.
My most-common frustration with it is when products I want to compare don't have the same quantity-denominator. (Either a different unit, or sometimes a different dimension like weight versus volume.)
I run Qwen 3.8 27b on each of my six AMD AI PRO r9700 GPUs at 80tps decode each and they cranks for days with 256k context, doing complex kernel, compiler, debugging, enclave, bootstrapping, pentesting, hardening, and systems work full time.
~$1200-1500/ea on ebay.
Also ~40tps on my strix halo now but with room for several sessions at once.
what point is there to be a human calculator if nobody rewards you for it?
and I'm not talking about wealthy Bay Area tech workers doing it so that they can write their substack blogs but rather the average person
If we're going to be the last generation that understands code at this level, we need leave it in a much better state before we die.
> then what is the path for one to become an expert so they can effectively use these tools?
The article briefly mentions guard rails and I think on the same note we need the AI agents we use for coding to encourage learning and comprehension in its users. Right now it’s completely voluntary and on the onus of engineers to put in the effort to learn since, as the article states, the friction that led to some degree of required learning is gone.
But what we're gonna manage the codebase in, then? What kind of language?
Will it be a natural language, or a formal one?
I feel that "coding" cannot collapse. Coding is just translation from natural to formal language. Somebody needs to write the specs. And writing/maintaining them in natural language brings a lot of fun - shifting interpretation, inconsistency, missing specification, etc.
I don't think it's progress for the field of SW engineering. But at least more "shareholder value" will be created.
The real issue will be with people that never learn it in the first place.
And even that isn't so bad, learning programming isn't some insurmountable hurdle. It took me less than 6 months from no knowledge to my first job.
I agreed with you up until this sentence. Entry-level devs with 6 months of require an enormous amount of time and mentoring until they're able to really perform well. I think the greater concern being expressed is not will people be able to type syntactically correct code?, it's will people be able to write software well?.
So basically I’m still doing things manually for everything that preserves my edge for interviews and as a technical person. But I use AI to generate diagrams and documentation, as well as operation scripts that no one is supposed to memorize.
[1] https://web.archive.org/web/20250623100834/https://ryanglove... (sorry for archive link, have a backlog to migrate old posts to my new setup).
We still have horse farriers for show horses or for family pets. It is a niche there for those that need it, but nowhere near as common a job as it once was. It will likely be like this. Those that like doing it will stick to it, but we will not see Indeed listing jobs for it as much. Of course with less demand, less will have the skills, so those that do have them can definitely command quite the power when they are called on for something.
In my experience human design -> agent code -> agent review -> release -> agent observer, does not (at all!) work. One absolutely has to be understand what is going on, how the system works, how the H/W works, etc, to catch agent errors.
The proposition of us losing those skills is terrifying.
Maybe AI will get better and we truly won't need a human in the loop. Then please also have all PRs signed by whatever model wrote them, and do not come to me for trouble-shooting.
Whether that actually has negative consequences is the more interesting question I think. Higher level languages collapsed understanding of CPU architecture and assembly level programming.
But as a result, we are writing more programs, not fewer, since we're operating at a layer of abstraction that matches closer to human thought process than what is dictated by the mechanisms of the underlying machine.
And this is because if companies are pushing you to "use AI and go faster", then there's no time to learn from GenAI tools, and companies don't care if you do.
Now everyone can be the boss.
at work, i can tell that people are not thinking and losing the ability to thinking hard.
I keep coming back to what LLMs "really" are. Recursive autocomplete, it keeps choosing the best fit for the next token. This really demystifies it for me. It also reinforces that is not "reasoning", so there's no hidden mental model.
its also wild how quickly and willingly many people handed this over. the apathy and desire for connivence runs extremely deep among my coworkers. we are at the point where people are not even running their local dev servers and clicking around to ensure their work is functionality correct, they just ingest the ticket, dump it into the model, commit the output, and say they're the most productive they've ever been.
the difference now is it is shamelessly encouraged.
one interesting topic is code reviews of AI code. code reviews have always been a bottleneck on the team and there's lots of trust and rubber stamping. I don't believe people are reading the code they ship.
I will say that it is a beautiful tool if you turn it on its head. if you don't quite understand something, it will never tire of answering your questions. It is a great time to need handholding.
Sounds like a good compromise, if not even positive from many points of view.
You treat the AI like a junior.
You make it write pseudo code in the tickets.
Then you make it write code in the tickets for the tricky stuff.
Have it reference relevant documentation/APIs.
Then there is no lost knowledge.
(Still, sometimes it sneaks in some "helpful" belt and suspenders, but for the most part, I know everything that is going on in the code base with this method.)
Thinking through a problem is hard. For the most people will take any chance they can get to not think. Myself included. I probably gain double digits of IQ when my internet goes down.
Do we think there is going to be a correction for the vibe coders?
Pardon I know such question is controversial but thinking ahead.
"I use AI to generate a summary which I print out and read slowly every time I use a new codebase" is the new propaganda line pushed by Anthropic and OpenAI. They cannot tolerate this rebuttal.
I think that is a question we are all mulling about these years.
Our expertise has been packaged into a 200$ product and the competition has become ourselves.
For the time being, a good understanding of programming and domain knowledge gets you much farther than without it. There might not be room for as many (professional) programmers as before, though.
Yes, this is the bet being made by everyone, together. No, nobody is actually making it.
The cost and benefit of a pipeline to train programmers is an externality. You do not benefit from training programmers as you do not employ them long enough to get a return on that investment. However you will hire some programmer hired by someone else and get the benefit of their investment. It kind of ended up as a wash, and there was enough 'shit shoveling' work to make it economical to keep some junior engineers around to keep senior engineers happy, and some of those junior engineers would learn and grow and end up as the next generation of actually productive engineers.
This entire system has already collapsed. Junior engineers are not low value and thus low wage anymore, they are highly negative in value, as before they would puzzle on something for days and finally come back with a small PR that kind-of worked but needed revision. Now they can puke out 50,000 lines of code that they don't understand in 3 hours. You'll never be able to make them sit and just do stuff by hand, and they can't supervise an AI. Having them around means you'll have to basically ignore them and not look at whatever they are trying to do, and never use any of the code, or else you will waste orders of magnitude more senior engineer time as they are forced to de tangle the mess to even understand what it does. Or you just go with it and in 3 months you have 25 million lines of code and no person or AI can make a change anymore without breaking 15 things.
There is no benefit to any company, at least in countries where you can't lock someone into a very long term contract, in hiring junior engineers anymore, so they will just stop doing this. If they did hire juniors it wouldn't help them, as they will still demand much more pay and leave if you don't give it to them the moment they can pass interviews at a more senior level. So they are just going to stop, the pipeline will dry up, and they'll pay a lot more for senior people as the supply gets more and more scarce. This is self reinforcing, the more you pay them the earlier they will retire.
By the time governments and the industry are feeling enough pain to do something about this salaries will be out of control.
This is asserted without evidence, and from a scientific standpoint, unjustified.
First, "detect patterns" makes it sound like a classification task, "is this a picture of a cat". But LLMs transform the input.
For example, an LLM can translate text between two languages while properly handling the names of the people described, no matter what those names are. That shows they are representing the text in a somewhat abstract way, that they can perform operations on that representation, and also convert it to useful output.
And, what I just described is the most general form of information processing algorithm. Science is not aware of any limitations in principle on such systems.
I am not saying LLMs have no limits, but "they only recognize patterns, and that is a true limit" is not a good argument.
Again, to be clear, this is highly dependent on the market, job, and hours and one needs to understand the difference between a CNA, RN, NP, etc. However, it's a regulated typically union job that's very secure and in high demand. It probably pays more than what you expect.
1. We still need to keep the lights on. If things go wrong, which they still do, humans have to debug it.
2. Jevon's paradox: now the expectation is that we ship stuff faster and there is still a LOT of stuff to do, not less.
3. We are still in the very early days of this and it is too soon for corporations to make a dramatic call like laying off 80% of engineering. We just don't know how that would look like.
we can vent here but it's not gonna change things, a wider cultural shift is needed. and even then I don't see it guaranteeing a "responsible govermment", e.g. see PRC China for example
Not surprised Germany values education a bit higher than us.
Also the reality is that modern CPU microarchitectures are so different from what old-school assembly programmers used that their supposed optimizations are likely to be counterproductive.
Nurses need to unionize I think and strike to get something .
I'll readily admit that they deserve it, I'm a lazy dickhead and they help people, but I think your blame is misplaced if you're insinuating that software engineers are to blame for nurses being paid less than ourselves.
We're already seeing this at the enterprise level. Companies have dictates from leadership that "if you're writing code manually, you're doing it wrong."
Okay, that kind of works for a while. We are indeed producing a shit-ton of code, but the reality is that engineers are pumping out code faster than the humans can understand and (honestly) review it. That sounds great until you realize that "hey Claude, read this Jira ticket and implement the feature in this code base" isn't really worth $200K/year.
This is all complicated by the fact that we're also losing our grasp on reality from the other direction because we have leadership air dropping AI generated manifestos on the product owners and product owners having to use AI to transmute all that shit into 1,500 word Jira tickets that are 10% necessary feature work and 90% LLM boilerplate.
So now you have software engineers whose job has changed radically to the point that the hardest part about being a software engineer is just filtering through AI generated artifacts from all directions just to try to get a feature out the door.
Nor even €50k/year.
Two things are true:
1) The coding part of my career is over. LLMs are capable of doing everything I've ever been paid to *write*.
2) My actual job also included non-coding work: Does this attempted solution even work at all? Is this solving the right problem? Even if it's a valid solution to the right problem, is it the best solution given the time constraints?
That last one, "given the time constraints", is a place where my experience is still useful. The AI is as lazy (or as optimised for fast wins) as the humans whose examples it was trained upon; but an LLM costs so little that the answer is always "do it right" rather than "do it fast". A lot of people don't know what "do it right" even looks like, having only ever known e.g. websites that take 5 seconds to load because of all the adverts and analytics, and never the world where machines with 1% of the CPU and no GPU at all could fit a fully playable first person shooter in the same memory footprint as that page.
At least, I hope this "is a place where my experience is still useful"; I may just be telling myself a nice story, same as all the other people through history who have found themselves obsoleted when the automation came for them.
But an LLM just happily chugs along and does it, no one feels the friction, which means you never stop up to think if you're solving the right problem, or in the right way. I'm already seeing this bite us in the ass, as you get hacks upon hacks.
Remember we used to spend enormous amount of time in school and in our spare time studying computer science? Algorithms, operating systems, compilers, and etc. All kinds of insights. All kinds of fun. All kinds of hard engineering. Yet, how much time do we really need to spend in our day-to-day work implementing or using the algorithms and etc that we have learned?
Engineers have done amazing work of abstracting away the hard algorithms and data structures. In the meantime, there has been little progress or few new fields in the past 10 years or so in business that ask for implementation of new algorithms. In contrast, getting LLM to work is a new field, so it requires tons of new implementations: KV caches, speculative decoding, all kinds of variants of attention like FlashAttention, all kinds of parallel processing techniques, RL pipelines, post-training pipelines, and etc. It's just that the field is so concentrated that only luck few get to work on them.
So, maybe it's not LLM per se that removes the need of writing code. It is the maturity of the software engineering that has done so. It's just that LLM fills the last gap: making knowledge transfer so much faster and cheaper - if all that's left for most of us is slicing and dicing of what has been already been implemented, then LLM can reliably take over.
This is not non-coding work. This is coding.
That is certainly not true. LLMs can't write code worth a damn still, and you have to babysit them to make sure they aren't doing stupid stuff. A human is still by far the best choice for doing programming, and when trend-chasing companies realize the damage they've done to their businesses they will stop pushing LLMs so hard.
took complex pub/sub event architectures down to the events postgres can do on its own, and a system architecture that works with that payload. am aware of the limits with this approach and stress potential on db
Unfortunately, this won't be captured by the metrics, because its not easily observable or instrumentable. There will be relentless pressure to use AI and more and more and prove that it is adding value. It won't add value, but whats available to executives is to lower wages and employment. There will be 'success' stories by most VPs that they've been using AI and here is proof (low wages + employment) and trillions of lines of code. LOC has always been a useless metric, but nobody cares.
> At least, I hope this "is a place where my experience is still useful"; I may just be telling myself a nice story, same as all the other people through history who have found themselves obsoleted when the automation came for them.
I don't think there will be more automation (at least not in a productive, helpful way), but a lot more enshittification and success stories of AI. Nobody cares about consumer experience or who is actually using the product.
Almost all CEOs/C-Suite/VPs are far removed from reality, they will do whatever is being hyped right now. Every Corp has had massive DEI initiatives in the previous administration and now anti-DEI in the current administration. Corps were doing blockchain nonsense and now they do AI.
Horrible if you got into this industry because you loved to code.
Amazing if you only ever saw the code as a means to an end.
I was just doing that yesterday designing a game demo lol
It doesn’t work for the vast majority of tickets he attempts because he doesn’t have the necessary understanding to even start thinking about if the solution that the autocomplete generates is even remotely workable. And that’s with fancy dev loops and whatnot.
The spacer between the keyboard and the chair still matters in my experience.
I have a hard enough time explaining why “yet another bespoke application on some unmanaged azure resources” is a bad idea when they have more reasonable alternatives at hand.
Now these goofs can (very nearly) press a button and do it anyway, with no comprehension of the consequences. It’s high fives and pats on the back, until I’m cleaning up the mess.
I guess that is taste?
Remove the dev from the equation. Suppose you need to produce circumstances for the slop machine to succeed on its own (imagine a loop).
What is missing? A good enough description of the current behavior, desired changes and how it should work? Understanding of the business domain or the system in question, its architectural choices? Enough guardrails and linters and tests to ensure that the solution will indeed be testable and work end to end? Cause a lot of the things that will help human devs will also help LLMs make their output more workable - for example ADRs in the repo and custom in-house tools to ensure they follow architectural conventions and practices (or just something like ArchUnit). Realistically, how close to a finished implementation must you get and how far backwards from that can you go before its output degrades too much?
I tried figuring that out myself on a personal project where I mostly want what I want in GitHub issues and then feed them to Kepler, which spawns OpenCode in worktrees and also gives me a UI to follow the execution. Most models, even SOTA, occasionally implement things wrong, but it's suprising how far you can take things when you've spent a dozen hours planning out what you want the end result to be vs not to be (talking about the goals for an entire system, maybe an hour of planning for a more bounded type of task).
Boris: "I haven’t written a line of code by hand in, I think, eight months now… Claude Code, 100% written by Claude Code".
Boris: "There’s no manually written code anywhere at the company… All of the SQL is written by models. Everything is just built by the models... Claude instances communicate with each other (e.g., over Slack) in autonomous loops"
This does not sound like they review the code either. So, either the frontier labs like Anthropic have figured out something that very few companies could replicate, or they are being incredibly deceptive. I don't know which is true.
1. Yes, we know, and can tell.
2. No, they haven't figured out anything. Just vibe coding it with their bleeding dege models.
I think that will be one of the most interesting things with models in the future. Even if we somehow 100% stopped dead where they are from a a model perspective, being able to run it twice and fast for half the price in the future will enable a hell of a lot more practical usage.
The kinds of business software I work on don't have those characteristics. If I needed something like a utils library, I think I could easily have Claude write the whole thing and not read the code.
Why would programming be any different?
That doesnt clearly apply here
This is so real. Jira tickets used to include a paragraph about what the change or feature needed is, a screenshot from figma for what it should look like and that's it.
Now I get tickets from the product manager that contain a whole novel waffling on about specific variables in the codebase that need to be changed which I have to wade through to work out what the prompt was that explains the actual product change desired.
When you factor in overhead and benefits, many companies were regularly paying that much for someone (many someones) to "read this Jira ticket and implement the feature".
We are currently in the "centaur" phase where a human-AI combination produces the best output, but I think some people are betting on the fact that the AI only product will eventually outperform the centaur. And with the cost of tokens falling thanks to fierce competition from the Chinese open weight models, it's definitely possible that those that bet on AI early will reap payoffs in lower expenditures for more output.
Honestly not sure which side I land on that bet but I definitely can't rule it out.
It's probably more accurate to call it the slop phase.
A bad or below average dev with AI will run your product and company into the ground in a matter of weeks.
Good: you can refactor your codebase at will, throw out legacy cruft by the megabyte, improve build/CI time, and simplify ruthlessly. Not to mention kick out new features in simple and consistent ways that align with what a user actually wants.
Or you could add megabytes of vibe coded crap, solutions that add a ton of mass but don't actually solve the problem at hand (seriously!), implement abstractions that are logically inconsistent with the rest of the system, etc.
Both of these are happening right now, and I think the latter is happening at a rate far higher than the former. But our fundamental dynamics are still at play - the ball of mud is still a ball of mud, even if AI lets you make it 100x bigger. The problem just gets more entrenched.
Eventually AI will learn how to simplify code, understand coupling, etc - and hopefully it will just iron out problems as it goes. I think we're a long way away from that. But this is uncharted territory, and I don't think anyone really knows. I certainly don't hear anyone focusing on that as a target of optimization however.
My hope is that we'll see a number of companies collapse as they scale - with basically no hope of rescue, and perhaps we can re-learn these lessons yet again. It kind of feels like GitHub might be the first example of this.
I had a recent experience with this. Although it was months and not weeks and there were other factors into play such as the market etc but the last company i was in had a poor engineering culture with inexperienced engineers equipped with AI output some of the lowest quality features for both customer-facing products and internal dev tools. A lot of customers churned, and im sure quality us part of the reason.
1. Execution of engineering task - Companies happen to focus on this mostly and this is the metric to measure easily. Using LLM tools gives a impression of improvement on this area which is what everyone is chasing
2. Growth of the Engineer - This is the one which was always a side product of company culture, individuals interest, work being done, time being spent to understand, learn from failures. A job being executed perfectly for the first time itself does not gives the opportunity for learning, no memories/experiences are built up mind of the person executing the job after a while.
The second part is the one which is under appreciated in current scheme of things since it was a by product. There is a concept of muscle memory which pretty much applies to everything.
The word "engineering" means doing things using a repeatable process to get predictable results. If you can get predictable results (e.g. guaranteeing the absence of data integrity issues, security issues, anything that will cause downtime, etc.) without looking at the code, you're still doing the work of engineering.
Your job as an engineer is to choose high-value problems to work on, solve them in the correct way, and guarantee that the functional and non-functional requirements are met. If you no longer need to read the code to guarantee that it has the right set of ilities, then I'm not sure that's obviously a bad thing, as long as the ilities you're enforcing result in a codebase that is sustainably secure and maintainable over the long term.
Perfect example of begging the question.
Vibe coding works great as long as vibe coding works great.
Personally, I'd love to see how you enforce security and long-term maintainability without looking at the code cuz we need to add that to our build pipelines ASAP.
That led to a reduction in knowledge of assembly in the average programmer but the people who specialize in it haven't gotten any worse at it.
The result was a generation of programmers who make useful software while very few of them understand the machine they program. You could easily make either a positive or negative value judgement about that result.
Yes, it's true... but who cares? We lost that skill because we don't need it anymore.
English is becoming a very popular programming language.
The nincompoops at the top think that if mountains of code can appear by snapping your fingers, they will be less of a problem.
If the mountain doesn't work, no problem; since it was so cheap to produce, just scrap it and snap you fingers again. The next one will be better.
Comparatively evaluating 2, 3, ... mountains of code is harder than trying to understand one.
And no, telling Claude to implement a Jira ticket is not worth $200K/year. Checking if it has not done something stupid and correcting it when it's trying to - is.
He would call you a moron.
He might even just fire you as a customer if you annoyed him enough.
My argument here is that what is worth $200k+ is the ability to distinguish the changes that must get thorough, critical review from those that need only a couple of specific things verified and those that require no manual review at all.
Our jobs have never been to write code. We’ve been saying for decades that LoC isn’t a rational way to measure engineering output, and now we have our chance to structurally change that system before processes re-solidify. In fact, I think the flexibility to adopt new systems and the experience and foresight to choose a path that is better than the status quo without throwing out everything we’ve learned is going to be what sets companies apart and makes individual careers over the next few years.
> we have leadership air dropping AI generated manifestos on the product owners and product owners having to use AI to transmute all that shit into 1,500 word Jira tickets that are 10% necessary feature work and 90% LLM boilerplate.
There’s an important criticism here IMO - the relationship between “business” and “engineering” is changing drastically. It’s going to be a challenge to set the expectation that just because Marketing was able to vibe-code a prototype in a day, actual implementation may well take weeks or months. Engineering should be considering things like security, scalability, and systems integration that aren’t a concern for Marketing - that’s why we’re being paid!
> So now you have software engineers whose job has changed radically to the point that the hardest part about being a software engineer is just filtering through AI generated artifacts from all directions just to try to get a feature out the door.
I think it’s an extremely heterogenous landscape right now. Where I work I’m struggling mostly with organization - keeping the (literal) dozens of inbound features that come in every day straight long enough to hook up an agent harness, review, validate, and deploy. At other companies the issues seem to revolve around their Agile-based processes. Or maybe it’s non-technical vibe-coders and their expectations. Or maybe it’s executive leadership flirting with AI psychosis.
Everything is in flux. It’s stressful and exciting, and I’m thankful to be around for it, even if I am in my 40s at this point and looking at the core skills I’ve built rapidly become worth exponentially less. It’s a huge opportunity for growth.
The problems:
1. The market is/has been saturated with engineers/developers earning well into the six-figures who are basically decent at writing code but not so good at anything else. So sorting out who is really worth these salaries is not going to be easy and the people who decide often lack the knowledge and skills to determine it.
2. You need to successfully make this argument to people who control budgets and more and more of them just aren't going to buy it.
I have over a 15 YoE writing software and guided coding sessions - that is, using an editor like Zed or VSCode with an LLM integrated, writing code how you normally would but using a flash model to prompt away the annoying parts and/or plan - is as productive as vibe coding, produces significantly higher quality, is actually enjoyable, and you actually stay sharp.
Flash models (DeepSeek v4 flash) tend to be so fast that you don't have time for parallel agents, you lock in and rapid fire prompts, building high quality software while incrementally reviewing it as you go. VETO bad edits and try again or rewrite them manually.
By contrast, I have noticed headless agentic coding tends to be an unreviewable black box. The major issues I've found is that, even with a human-in-the-loop, you accumulate defects which compound and eventually you're spending millions of tokens to make trivial changes in a ridgid codebase.
Ultimately, tiny, highly cached, fast models like Qwen's 27b/a3b range or DeepSeek flash are highly capable and relatively inexpensive to run. Hoping people realise we don't need 14 trillion parameter models and I'll be able to buy some ram for my workstation
The subtitle of the story tells it all.
There are some people who seek out friction. Think about an athlete or a hardcore nerd.
The best engineers are ones who were fascinated with computers and learning as kids and persued it at every opportunity. Found their own friction in other words.
For those kinds of people, friction-seeking is the constant and what LLMs did is moved the point of where the friction occurs.
For example - the best engineers I've worked with didn't necessarily have lots of experience coding in assembly because that kind of friction was no longer necessary. But they could solve hard problems (and if a problem really required assembly they could go learn it)
What I think will be hit much harder by AI is the low tier engineer. Someone who was never truly curious and committed to it, for whom it was just a job. For example a typical offshore ticket pusher kind of person. That kind of person never went out to find friction and that's the kind of thing that's never going to fly again - if I want mediocre or average, the LLMs are sufficient
Completely unsustainable.
I reckon everything needs another 6-18 months to ripen, then those of us who haven't bet the farm on slop will be in big business contracting and consulting to drag companies out of the pit they're digging themselves.
I've been worried about this a lot, I even created an agent skill called do-i-understand that's designed for novice devs (and experienced too, because atrophy) where the LLM asks you questions about the PR you're about to submit. I've found it helps a lot: https://github.com/AnthonyPAlicea/skills/blob/main/skills/do...
One way or another, there will be a skill reckoning.
First and foremost, it's an issue of "dependency": if you stop training the "muscle" of logic and reasoning, it gradually atrophies, just like unused physical muscles. You become dependent on external tools that replace a capability you once had yourself.
A historical example that brought about a similar shift is this: when the production process moved from the craftsman's mind and hands to the Fordist factory (and the assembly line), the skill of building things shifted from human craftsmanship to anonymous, structured processes.
Bit by bit, traditional artisans lost their knowledge and "know-how." Today, having a piece of furniture in our home depends on a massive production and supply chain; the "average" person no longer has the ability to build it themselves.
The exact same thing is happening to software.
We are the (now "former") software craftsmen.
Horsemanship and sailing were both specialized skills of high value to society, and now they're not. But in each case there was probably a liminal period, when being an accomplished horseman or sailor was still valuable, even as motors were taking over. Eventually that period ended, as the new generations without those skills found ways to get by with cars and motorboats.
I could wax philosophical about what that leads to but enough people already have.
This is speaking from my experience as a systems/c/c++ guy. If you are a js web frontend guy, python, or whatever I have no idea if this applies to you.
Think about the worst colleague you've had, now give them a tool which 10x's their output :)
now that's a phrase I have only heard in one other context: as an alleged quote from former Boston Celtics player Isaiah Thomas (who was really about 5'7 despite his official listed 5'9; yes, professional basketball player in NBA). This was around 2017 when he had just come off a career year and was soon due for a contract extension in the low 9 figures. He got injured and bounced around for a few years before eventually leaving the league, and never got that payday.
Sure there might be a dozen individuals making over $300k but most COBOL devs live in the deep south making less than $100k for their entire careers. Fine salary, but that line of work is severely limited and the salaries do not reflect how valuable that work is.
So, partly agree, partly disagree.
Writing code in programming languages is a skill/necessity created by us to instruct computers what we want them to do
Initially in the 60s, this was done by connecting circuits one way or another (think ENIAC). Then we devised "programmable" computers and devised a bunch of codes (computer code instructions) that abstracted away those cables.
The we created Programming Languages to further abstract away the hardware complexity, and to be able to write down our wishes in a way that is more transferable between us humans, but that is still computable by machines.
But with LLMs and neural networks, at some point these abstractions wont be necessary.
The computers will still be making computations, but the way we tell them what we want is going to evolve.
It's fascinating.
It’s true we usually don’t care about all the details, which is why abstractions exist. An abstraction hides details by fixing them, and lets you specify the rest precisely.
An LLM isn’t an abstraction in this sense any more than asking a coworker to do something is. The details are not fixed, but decided for you. If you can’t understand or modify what was decided yourself, the only interface left is going back and forth in natural language.
So no, the abstractions won’t become unnecessary. The details don’t just go away. Plenty of people really don’t care about them, and for them the LLM is fine, but only because it’s gluing together the millions of LoC of existing libraries and frameworks where the details have already been fixed, overwhelmingly by humans.
There will always be a need for human professionals who aftually understand all that obscure stuff, and per TFA, it’s looking like it won’t be the ones who went all in on AI.
I agree that AI reliance is hurting engineer's language understanding, but I don't know how much we should care.
I'm sure I'd still be able to do it, but was saddened that I didn't get it as quickly as I think I used to be able to. Atrophy might already be in play.
And there are huge amounts of time, energy, and writing sunk into arguing about AI or trying to convince people of the danger or problems.
Sadly all of this is taking time away that could be spent on real progress and learning about coding/FP/PL/algorithms/type theory/etc.
Well we could also put a halt to the production of tesla and go back to producing the perfect vehicle, the Saab 900 Turbo, but you know, I don't get to (yet - I will be King one day) determine what others do
It might be like how an archeologist since the invention of plastic can date the period of the soil as post plastic, if one could cut open the software stack of the coming systems that will be built in the near future one would could data that code as post agentic LLM as programs are going to be mostly bloated ad-hoc, poorly thought-out and patched in a way that does not concern it's self with the correctness of the algorithms or data structures chosen.
I am not Anti-AI but I think it's going to be interesting and I am surprised at how bad a program ends up when one tries to "vibe code", though often ends up working, though in isolation and when used with discipline (that the tools themselves psychologically make it harder to do) can produce some very good code and being able to use loops to solve difficult problems or problems that simply would have require banging ones head against the problem many times is very profitable.
By way of example I use Kdenlive 90% less and just use Codex to issue the ffmpeg CLI commands directly when I want to crop/format video on my PC.
This is an accurate description of 90% of corporate software pre-LLM.
LLMs are introducing a unique kind of bloat and dysfunction but it's not like they are replacing pristine code. Most software that is subject to market pressures and is generating a profit is painful to look at and difficult to work in.
Now with agents I don't have any insights into what its doing with kubectl and would have no reason to learn how it works (other than my own curiosity).
I agree that the human no longer gains expertise, but I am not sure that will matter in the long run.
Local AI would thwart this vision (which is why they're desperately also trying to ruin the hardware market at the same time), but the effort it would take to get local models to run as well as these frontier models is an odd thing to pursue just to avoid learning the fundamentals and gaining direct experience.
While I agree Local AI would be a good goal, even the huge open weight models that can't be run locally are already thwarting the US AI lab's duopoly. Other companies can and do serve up these open weight models, and without having to amortize R&D they are able to serve the tokens at a pretty cheap rate.
In some ways, it feel like I'd be "stuck" at my 2024 capabilities if I didn't push myself constantly, and I'm really glad for every hour I spent learning/applying new stuff before that, because now it feels way harder.
I also see no evidence that AI programming is a difficult skill that cannot be learned by any intelligent person in much shorter amounts of time than previous professional skills required.
They've become a meat proxy[0] prompting LLM to root cause for them and the ability to fix is capped by LLMs ability, not the engineer's.
I've tried to convince management before this happened and yet here we are.
I have such a hard time quickly reading pages like TFA. Headings, bullets, line-spacing, width, link hover animations, even the font. Even reader mode defaults aren't great for me.
Compared to, for example, https://www.paulgraham.com/best.html
If all your peers are only accepting what LLMs tell them then you should do the opposite. Spend some of your time writing code by hand, especially when working at a new layer of the system or on new things you've never looked at before.
While the agents churn out more CRUD crap in the background spend your attention learning how everything works.
I do way more experiments with way more things by taking advantage of LLMS. When I have them do research I use that as a starting point for my own reading (but don't blindly trust them).
I find I am learning more faster.
Change is an opportunity. Use it to your advantage. Let other people while and moan while you get things done.
People are distrusting whole swaths of concepts because they do not understand how things actually work.
Like going away from zerotrust architecture to holding open a long polling connection.
At least I seem to remember that long polling wasn’t that great, but i start to even doubt myself.
Anyone wondering how the Roman Empire was wiped out? I guess we will find out soon
There are project I've built from scratch that I would feel confident to hand off to a bunch of non-technical vibe coders and I know they would be productive and the product would likely be secure; because the existing codebase already exhibits all the patterns and principles that are required for that kind of project.
It would probably slowly degrade over time if a lot of vibe-coded logic is added on top but I think they could get very far feature-wise whilst keeping the software reliable.
But even though the value of such codebase has increased, people haven't adapted to this new reality. People are generally not good at telling what is good code. Because we don't actually have consensus on a definition. My definition is that good code is code that is easy to extend and maintain.
If implementing a feature requires a huge amount of tokens, then there's a good chance the codebase is not great.
I've worked on a codebase where a small feature requires might require 3k tokens, but on a different codebase, a feature of similar complexity would require 30k tokens minimum... And it's not about the size of the project; it's more about how the logic is divided and the architecture. And importantly; it's not a one-off; it's a clear observable, repeatable pattern.
This has an issue in practice.
As an example, Function Programming (FP) has a high constraint on programming, where some don't even have loops (gotta use recursions all the way), and almost no state mutation (Exlixir I believe have a mutable state? forgot).
FP sounds great with all those constraints. But what about the adoption?
It's hard to switch your mindset, and not as natural, thus long leraning curve, and not as wide adopted <- productivity goes down as all others need to know and use it well. That's why still imperative and OOP languages are ruling the world.
Same for these constraints. The author promoting using AIs only to subsets (no coding? wtf) is something the majority won't follow. Yes, the constraints sound great, but at what cost? By the time one learns everything, and everyone moves along with AI building stuff fast and cheap, they will be left behind.
---
I do not like the author trying to make his points authoritative by using Halo effect using quotes from the industry and could have had been taken out of context (That's where AIs mess up the most, they only get chunks of text and lost context of before and after exerpts).
LLMs don't particularly care how hard the type system or borrow checker is (as long as it has good error messages that can guide it to a solution), they can grind away at these things, and what you describe as a negative actually becomes a strong positive. I think we'll see strongly typed systems plus some level of formal verification become popular as an AI-preferred language.
When the burden of review is lowered because you have purely functional building blocks, with lots of formally verified chunks - people will choose that simply out of (productive, beneficial) laziness. Not to mention we're in a ram crunch, and being able to run your stuff far cheaper has an immediate financial incentive.
I really treat them as "delegation layer". They're systems that sit right alongside your stack, and at this point, at every level in it, like a connective tissue that intersects at all points. I can delegate a spec, a function, a file, a whole feature set, an app, a debugging loop, etc.. My day-to-day isn't all that different, it's just like having the "Ship's Computer" that I can ping anytime I need something. I'm enjoying the benefits that a smart data processing pattern interpolator provides, but I'm not giving up any personal control or agency. My goal isn't to go faster, nor is it to avoid coding, because coding isn't drudgery or below us just because we have new code generation tools.
Because I was using DeepSeek flash, the interviewer was confused at how fast I was getting edits - he'd only used Claude and was used to it thinking for 40 minutes to update a function.
We were building a frontend application and I was able to do that whole guided prompt/review loop. I finished the task with no defects (high quality code) and way ahead of time.
I could tell the questions he asked were prepped for vibe coding with slow frontier models, like "how do you review the code to ensure it is safe to commit" - "make small/reviewable edits and read the code the LLM emits?".
Despite meeting the criteria of a principal engineer in the other rounds, the feedback on the AI round was that "I didn't delegate enough to the AI and was too hands on" lol.
I agree with your last statement. Frontier models are nonsense. We should be concentrating on focused models that are cheap to run, and don't require destroying the SBC industry, and making my hobbies financially out of reach.
* The scope is constrained and clearly defined
* You articulate an exhaustive and objective set of success criteria
* You provide a deterministic tool for the AI to evaluate its output against those criteria and receive feedback (compiler errors, unit tests, regression tests, etc.)
You're essentially writing a comprehensive spec for the AI to follow. The time suck was always from making the spec and the QA bulletproof.
One shot benchmarks that have exhaustive prompts are impressive, but benchmarks that emulate real software development (iterative changes over time with changing requirements) have close-to-zero pass rates.
I've been trying to map the LLM advancements and the current state of software development onto prior technological improvements. History is littered with similar cases where the abstraction layer ends up getting lifted, and people struggle with getting accustomed to working at that higher abstraction level.
For the people that fall in love with a single abstraction layer or don't have an interest in learning new paradigms, when their known pattern is abstracted away, they're condemned to being left behind, either unwilling or unable to adapt.
I don't think any industry is free from this, any person in any industry/profession over a period of 20 years or more has likely had to undergo massive adjustments as technology changed their field.
We're not unique, but that doesn't stop it from feeling so jarring when it happens to us
But there's no abstraction layer that ends up getting lifted. When I use a library like SDL or a standard like POSIX, I don't tend to look at the underlying implementation. Instead I work with the high level concepts that they come up with. There's no such things with AI tooling. The most similar is when fully vibing software and everyone knows the quality of the result.
I've learned something at every abstraction layer in computing from electronics (hardware), theory of computation (software) to high level programming languages with their paradigms. Same with several domains embodied by libraries. LLM tooling is more like shamanic ritual than engineering.
Who else is going to talk about a topic like how Searle would approach differentiating ownership from privilege in RBAC?
[1]: https://en.wikipedia.org/wiki/Argument_from_incredulity
Of course, bills need to be paid and it's not all that simple. But at least there might be such a silver lining.
If I use two LLM's to create some chunk of code and they both do it slightly differently but they both compile and pass appropriate tests.. it honestly doesn't matter if the LLM itself is not deterministic in exactly what it's going to output.
I would also argue- doesn't that make sense? You give two human coders the same task and they are also going to come up with slightly different results.
Saying "as long as it works and tests pass" suggests that we can test for every possible scenario. We can't. And tests can be flawed on top of it. Which is why an LLM is no more a compiler than a human coder (as you say) is.
LLMs are currently capable of level 1, but not capable of level 2. Trust follows what is actually deterministic.
This is why in order to understand new codebase or to ramp up to new projects, I use AI to generate a textbook style reading material for me along with "verify yourself" types of exercises along the way. Then I print them on a paper and read it using a pencil/pen and take notes.
Because of my math training, I am in the habit of slowing down to read textbook style texts which helps.
This is fascinating. Any chance you have your workflow or prompts documented and would be willing to share them?
For a bit of extra unease, consider that these people will retain their right to vote, both in making business decisions as employees, in shareholder meetings as shareholders, and in government elections as voters, despite their atrophied reasoning abilities. Who will wind up whispering what to vote for in the ears of these reverse centaurs?
Tangential, but this is one of many reasons that I (and I suspect many others here) have taken up wood working.
> Bainbridge argues that new, severe problems are caused by automating most of the work, while the human operator is responsible for tasks that can not be automated. Thus, operators will not practice skills as part of their ongoing work. Their work now also includes exhausting monitoring tasks. Thus, rather than needing less training, operators need to be trained more to be ready for the rare but crucial interventions
(Not to say that the current state of the art with LLMs merits borrowing money for a multi-trillion spend on hardware that will be obsolete in 5 years, but for me that is a different issue).
The master decrying the invention that resulted in more wisdom for more people than anything prior:
"For this invention will produce forgetfulness in the minds of those who learn to use it, because they will not practice their memory. Their trust in writing, produced by external characters which are no part of themselves, will discourage the use of their own memory within them. You have invented an elixir not of memory, but of reminding; and you offer your pupils the appearance of wisdom, not true wisdom, for they will read many things without instruction and will therefore seem [275b] to know many things, when they are for the most part ignorant and hard to get along with, since they are not wise, but only appear wise." (quoting from https://www.historyofinformation.com/detail.php?id=3439)
Plato (who actually wrote these quotes, because Socrates only spoke) wasn't against just "writing" in a general sense, or that he felt it was going to hold humanity back. He was against treatises and felt that someone could "memorize facts" without having to actually think about them on a deep level. Which I think, ironically, is not all that off the mark especially in the age of the internet and now LLMs, right? He also felt that memory could decay if we relied on written facts instead of having dialogues, which again, not all that off the mark, either. He underestimated the compounding capability of technology and our ability to record data and information, but people remember a lot less these days than they used to because we'd largely given up that ability in exchange for the instant information machines.
Of course, he's just one man (or two, if you consider he was conveying Socrates' thoughts, as well) who lived in ancient times and couldn't possibly foresee how technology would evolve...but when I went back and read his concerns, it was ironic to admit that much of what he warned about still applies to this day. People are far less informed despite having access to more information than we've ever had because we've exchanged the ability to remember for the ability to just look things up. And, we also have less meaningful dialog than ever before; people just sit on social media, copying and pasting "facts" to each other, instead of having actual productive discourse. When I look around, the world doesn't seem brimming with mindful critical thinkers and there's reasons for that (many reasons, of course).
AI tooling can be a boon to learning, but it requires us to stop using them for code generation as a primary purpose (at least for juniors) and instead advocate for Socratic workflows that still require manual coding practices. When someone decides to code something, it's not just learning syntax; it engages a variety of mental disciplines from critical thinking to planning to creative problem solving to logic and math, even.
Those skills are often taught in an idealized situation like very small projects. In the real world, there are a lot more practices and techniques to develop software in a pragmatic way. Those "engineering practices" are much more useful than the above, unless you're in some specific domains.
This is the key point. Should we stay vigilant and review all the code, ensuring we have a decept grasp of what it does. Or just green light everything and treat the code itself as a black box. I would hope we do the former, but many are pushing for the latter.
Example from my workplace: pull requests that were already too long before LLMs by a few hundred or a couple of thousand lines are now several thousand lines longer and they get opened much more frequently. The primary source of these PRs is a well-respected senior team member with a ton more experience than everyone else by a measure of decades so nobody dares tell him to stop. Other team members aren’t far behind in the PR opening rate and admittedly I am one of them, but I try to mitigate the issue by doing multiple LLM reviews and anticipating when it’s ok to just approve it myself (have never had an issue in making this judgement thankfully). Part of this IMO is almost certainly because there’s now peer-pressure within the team to churn out a ton more code and that’s adding to the already clear message from leadership that we need to leverage AI to deliver things at unprecedented levels.
My point is that it’s not just that some people are pushing for greenlighting everything without due diligence, it’s also that they’re pushing others to toward that outcome with their extreme AI enthusiasm which they don’t see the ramifications of or just don’t care.
It's a similar level of excitement as when we started doing devops stuff before devops even was a thing (which has been one of our main things for a very long time now).
Horrible if you got into engineering because you wanted to solve problems.
Amazing if you only ever wanted to see the problem solved.
No, but there's now more people who know how to engineer reliable ships then ever before.
In 10 years, how many people will still understand the codebases they work on?
To be clear, I feel the same way as you. I have been doing this for about twenty-five years, and I have developed a keen sense of taste and judgment to know how to properly scrutinize and utilize these systems. I definitely feel like I am able to learn things faster, but that is directly correlated to having experienced a lot of friction over the years that cultivated said taste and judgment in the first place.
I imagine it's sort of similar to someone who's just getting started in mathematics, but is introduced to WolframAlpha, versus someone with decades of experience in the field and what they learn with it.
Would recommend your senior team look into stacked PRs, which was made to handle this scenario exactly.
But there are cases where there are no changes, or only backwards-compatible changes. Take the core WebAssembly specification, for example. Extremely detailed, formally verified, comes with a large test suite. Someone else has done the hard work of specifying the behavior of every possible edge case. A WASM runtime implementer can point their agent to that spec, plus some prior art. The remaining manual work is in constructing a test harness and collecting a corpus of test WASM binaries that exercises decent coverage.
I'm sure you were saying "bleeding edge" but my first impression was to translate dege to mean degen/degenerate which made this so much more fun to read.
If you worked in SV like I did from 2010-2020 you know that the exact opposite of that will happen. I saw hundreds of companies successfully scale out of their garbage stack such as Facebook and PHP.
The only thing that matters is the problem you’re solving and the quality of the code is irrelevant.
I think much of what we see there is really heroic efforts: subsystems get swapped out with things that suck less, PHP gets a new optimized runtime, etc. Basically - unsung people do the hard work that rights the ship, meanwhile 10x more people are adding tons of crap and it barely balances out.
Now that 10x is more like 1000x - and if we don't use these tools responsibly, I think we'll see the costs outweigh any kind of benefit.
Or I could be completely wrong. Worse might actually be better. People can vibe up a new replacement thing when the original thing becomes unmaintainable. That feels wrong/gross to me, but maybe I'm just old fashioned (like the old folks from Inmos/Symbolics/Transmeta were).
It reminds me of property based testing. I really liked the idea when I heard about it, but most of what I work on does not tend to behave in a way that can be easily described mathematically like that.
Maybe I'm just not clever enough to do it.
This is the same “but the ai can’t even do fingers” argument all over again, and it will age just a well.
The whole vibe coding thing only really started taking hold with the release of opus 4.5. That wasn’t even a year ago. you absolutely can be quality software without looking at the code. The only thing up for debate in my mind is how big that software can get before you need to look under the hood.
But that’s dictated by current model capabilities. And I don’t see any reason to believe that they will suddenly stop improving on this front.
Yeah, a bespoke program that does exactly what I need it to do, at a speed that I had forgotten was possible on computers, with customization that is an exact fit to me, at a cost that is smaller than a rounding error.
I get that LLMs struggle with the old paradigm of a single piece of software meant to serve every conceivable use case of every conceivable user, but I kinda hope that paradigm dies.
AI isn’t “a code abstraction layer” any more than Jira is, or the engineer themselves is.
Imagine for a moment that, rather than using a compiler to translate c++ into assembly, that it instead has to be done by a person on the team. If that were the case, the resulting assembly code would certainly have to be reviewed and assessed before it was accepted.
Why?
Because people can also be leaky, non-deterministic abstraction layers. The only reason that the output of a compiler isn't regularly reviewed (in 99% of cases) is because it's extraordinarily reliable and consistently correct, or correct enough for most cases.
It's not terribly dissimilar from delegating engineering tasks to other engineers. When I ask someone else to develop one component of a larger application, I'm not telling them exactly what lines to write, I'm giving them some kind of structure and they're filling in the rest. The communication to them is above the implementation layer.
Agents are increasingly letting us work at that same boundary. Just because the current state of LLMs requires engineering knowledge to review the result doesn't mean an abstraction hasn't occurred. It speaks more to the current quality of that abstraction than the absence of one.
I think that if LLMs were able to achieve the same consistency as a compiler, most people wouldn't ever bother to check the underlying code it produced. I also think it would be difficult to not acknowledge that LLMs have gotten better at converting natural language into functioning software. The abstraction is certainly not perfect, but it is clearly improving.
I'm trying to write a novel, have been for a while now; one thing I hear from professional writers is that one should only do this if you are prepared to re-read whole thing six times before you even send it to the editor, and when you do send it to the editor, you and they will likely spend several months repeatedly re-reading the whole thing as you respond to a long list of changes they give you.
I do not think anyone could do the job I do with AI without my decades of experience, is my point. For now.
The AI programming meta that keeps evolving (last month it was loops) seems like a relatively trivial skill to adopt compared to learning what it’s like to do high quality work and to participate in a team, and these kinds of things. If the AI stops improving then everyone will master the AI skill set that everyone else is using, and people will acquire maturity and experience over time, like always. This is my point.
Because great insight and expertise stems from foundational knowledge.
I am a huge hockey fan. In the NHL, the players do not stop practicing the fundamentals once they make it to the NHL. They practice the fundamentals even more. Many practice the same drills as youth leagues -- stick handling, passing back and forth, shooting, edgework, shooting, rebound control, etc..
The best of the NHL might not hit the hardest, have the most accurate shot, or skate the fastest. What separates them from the rest of the lot is that they are fundamentally better than everyone else.
But that isn't the real problem here. Computer Science was always a theoretical concept, really. The meat of the issue is the rug being pulled out from the juniors and mids.
Reading code and writing it are two different skills. I believe that both are needed to maintain a codebase. It is much, much easier to understand what a service is doing when you're actually writing code that supports it, just like it's easier to remember the contents of a report that you wrote versus one that you read.
With AI, people can build and collaborate on applications much more complex with much less technical knowledge. It might be ugly and crappy but I bet they’ll be mostly autonomous and not need to work through their IT team, or go through the hell of PM and requirements. If I know my requirements, I don’t need you. Hell, I can just start building and add requirements as I come across them. It’s not a major risk to the project like it used to be.
It will, and is, going much faster than the spreadsheet did.
Transit routing is still moving fast in this area. We're still figuring out the best ways to return thousands-by-thousands transit time matrices with query-time truck dimensions and traffic updates. It can't be the only field!
Not to mention we told everyone to not roll their own anything, just use the free library for, clocks, time, crypto, auth, IPC, etc!
The business had pretty questionable (read: bad) software for a long time, then came under new management a few years ago. I joined just under two years ago and since then we've been modernizing the code with an eye to move it off of the in-store servers its running on currently and into the cloud. That is complicated by the fact that the franchisees are not used to certain things, like really needing to have reliable internet with a failover, so we've had to (or had the opportunity) do some serious hardening on the in-store servers and how we deploy software to them. We've switched them all to immutable OSes, added a bunch of telemetry, and added some new services that are slowly eating the old ones' responsibilities and will be much easier to move to the cloud.
Also the winters are ok but not as nice as they used to be. I grew up here so I remember when they were extremely snowy and I miss that, now they're fairly grey and mild.
:)
Even their fans agree, LLMs are inherently unreliable. The fact some people trust them regardless is due to a deep flaw in human psychology that I believe has not been significantly exploited by any previous tech. There will be tears.
1. Decades of generally reliable systems (I trust my calculator because it always says 1+1=2) has trained people to believe what computers say.
2. LLMs "speak" with great confidence, which has, as you say, a psychological effect.
> Introduce me to these concepts layer by layer with a textbook style introductions.
> Use color coded boxes etc. just like in textbook to highlight important details.
> Along the way, give me exercises to do myself that would test my understanding so far.
> Put everything in a local HTML file that I can open in my browser.
> The entire textbook should be printable so format accordingly.
That's about it.
The HTML part is important. Earlier I used to document in markdown format, but I found claude/chatgpt does a far better job with formatting and colors in HTML instead. Plus, you can open the HTML file in a browser and refresh when it makes changes.
Another thing: the HTML is built gradually. I keep asking AI some follow up questions and I instruct it to keep the HTML up todate.
the real feedback they couldn't say is that you were a threat to his position.
I'd wager you're far better than whatever candidate they landed on, because the person who apparently qualified for that role probably isn't thinking while they prompt away.
Why are engineering teams looking for brainless monkeys? It's to their detriment all around if they aren't hiring capable people.
I think it depends on the - ahem - context. In my experience they usually can, sometimes they can't.
But generally their sense of software architecture remains abysmal, so even if their writing is ok, you need to have architectural authority the very least to guide them.
They wouldn't be able to verify if the code is correct, but based on my testing, if the work is simple enough, it probably wouldn't matter much.
There are a lot of companies whose modus operandi is basically "build the same basic site 300 times, except with a different colour scheme and logo".
;-)
Or, slightly changing this. The source text needs to speak the correct vocabulary and language to produce a good completion. See the chat where Terry Tao is doing maths with an LLM. There’s _no way in hell_ I could get to his output because I just have no idea, and can’t speak the language.
Same with any field.
The edge of correct/better when facing ambiguity is very fuzzy, all models from the past 6 month or so have similar random ways of spinning between too-literal avenues and oddly misplaced misled fixations. Taking the right initiatives in face of uncertainty is definitely AGI, and its not there, and perceptrons + attention layers just ain't got what it takes no matter how hard you push.
Knowing what not to do has always been important, but it is perhaps more important than ever.
Aka “actual deep understanding”.
I have a design principles + tech preferences doc I force llms "lint" their approach against. It's not perfect but it helps. I call it a bias field, pushes them toward hopefully the happy and harmonious (with the rest of the system) paths. Obviously this is only partial and imperfect enforcement, but if it's applied to everything consistently it naturally encodes some self-consistency and harmony.
I’ve done the same but it’s a moving target as models advance and I find half of my points are ignored until I’m prompting “No wtf why are you still trying to symlink the global Python executable just use the virtual environment that’s already activated”.
Anyway, companies are pouring billions into improving AI tooling user experience so most of what I do manually I just anticipate to be a waste of time. There’s no way my hobby fiddling will outpace whatever gets released in the next couple months.
In the meantime, real linting does work pretty well, if you can write a detector for whatever antipattern you find LLMs fall into (like multiline comments).
I get patches into the biggest AI projects all the time to make inference faster on my 3090 and all you have to do is read the contribution docs and open a small PR.
It’s not a cabal of super geniuses. We are literally in the Wright Brothers era of AI.
You can trivially outpace what big companies are doing it’s insane, why do you think so many innovations are coming out of scrappy Chinese labs? They are willing to engage instead of being defeatist about it.
Although, given opus 4.8->5.0 experience within claude code, I don't know if I'd agree they necessarily improve things with time. But that's just my experience. I should probably read their release notes, maybe just a tweak to my claude.md would help
Not my core competence and I’m not interested in making it a core competence while the field is still rapidly changing. I find harness engineering interesting, but not lucrative enough to push one of the many very important other things on my plate to the backburner.
I don’t think Python language developers are super geniuses either, and I’ve never submitted code for consideration.
Often they can't figure out how to test the thing, so "does this attempted solution even work at all?" is its guess from reading the source code, and sometimes I've even caught them writing "tests" which are a regex on the source code, not functionality.
They know almost nothing about "is this solving the right problem?", they're trained to assume the user is right, not to frame-challenge.
My general experience both in game and non-game projects is that it will be lazy by default rather than solving things correctly. Sometimes I spot this from their responses, other times I only notice with manual testing.
The point is that AI is aware of constraints and can manage a round it. You've got to keep in mind the size of a typical software project and plan. Most people aren't writing a kernel. They're writing some backend tool and need a project that fits into a couple of quarters with a handful of people at most involved.
Only if you instruct it with constraints. Otherwise, it's happy to implement whatever workaround it needs. But it still takes a dev to know what those constraints are and why they're needed.
The LLM will look through the codebase, think, and tell you what it would do and if there's any design decisions you would have to make, as well as other things you probably need to be aware of. Then you go through the LLM's output and address all those decisions point by point, asking the LLM more questions if something isn't clear or requires more investigation or you're unsure what to do. And end with "address what I wrote, and share any other thoughts or questions or things to clarify you might still have, don't implement yet"
And you do this back and forth until all the design decisions have been addressed and you feel confident of what the code and architecture will look like, and only then say "ok, implement"
That way you get a lot of the benefits of writing it by hand (being forced to think through what the best design would be and how it would integrate with existing code, and increasing understanding of how existing code works) but it's still much faster. The tool I personally use is Cursor in auto mode.
P.S. actually before even that you first ask the LLM "what is the current state of X in the codebase" and then you ask follow-up questions until you have a good understanding of all the details that are relevant to you. And then you can start having the design/implementation conversation in the same chat context, since having the above information in context is useful.
P.P.S. and you can also ask a bridge question like "can XYZ be cleanly added with how things are currently structured?" or "what would it take to add XYZ to the project?"
I find AI agents really great for codebase exploration and understanding how it works. In some ways even moreso than from manual implementation since it's easier to get a global picture.
You can also ask it questions like "are there recurring patterns of how X is done in the codebase?" to which it might answer sometime like "there are actually three competing patterns" and tell you what they are and the exact files, and then you can choose to refactor them if you want.
https://www.aihero.dev/skills-grill-me
https://github.com/obra/superpowers
When I'm doing something quite complex I'll also throw in gstack office hours but I find that its usually overkill for many things
https://github.com/garrytan/gstack
I do often use the /qa skill regardless though. I always hated QA because it was so monotonous and it required a level of organization and creativity i never had (i mean im the developer so if I thought of edge cases id have solved for them)
I like that /qa covers and documents every edge case that it tested and I can nudge it about other ones
- Functionality which exceed the technical ability or knowledge of the developer which built it; and
- Functionality which would require an excessive number of changes that time/cost became a constraint (likely because it wasn't an incremental change but a significant rework, or just a bad fit with the existing product).
Sometimes I had the displeasure of joining a company where you could see developers hit these blockers and tried to fight through them (likely under pressure from management) rather than avoid them. And in the process they created a mess of buggy, half-functional spaghetti code which someone else later had to resolve.
Today however, people can use agents to smash through these blockers and ship an incomprehensible amount of crap. And what's worse is they're celebrated by management who don't know any better and see this all as a productivity win with no downside.
I suppose there are two ways to look at this – some would argue that engineers don't need to understand what they're building in detail anymore so non of this matters. Instead they can always use agents to explain what's going on and prompt them to fix any issues that come up.
Then there's another camp which might argue that agents will fundamentally have all the same limitations as humans, and that at some point a codebase will grow too complex that it exceeds even the limits of an agent's knowledge retention or intelligence. Or a codebase may just grow so large that it costs 1 million tokens to make even a simple change.
Unless agents start saying, "there's no way I'm shipping that" like humans used to I don't really see how we avoid the latter scenario... Complexity simply must have limits even if agents allow the bounds of those limits to grow.
At some point models will have to take control or we risk hitting these limits in irrecoverable ways since complexity far exceed that of what a human can reason about well.
If this wasn't true then "build me all the software" would be a reasonable prompt. Because why wouldn't we just get an agent to build everything we could ever possibly need? It's obvious that in the limit there will be limits in knowledge, intelligence and cost.
Whether it's humans or agents, someone needs to manage complexity. That is the most important thing a good SWE used to do. It's why technology selection mattered, it's why good architecture mattered, it's why clean code mattered.
One theory I have floating around in my head is that if a whole code base was microservices and micro front ends that were all less than 5,000 loc then you could fit the whole thing into a 100k token AI context window when working on it. And being few lines of code would force them to be somewhat simple
I've had a lot success at the company I work for shipping small contained microservices for less technical teams to maintain with agents.
Personally I think this most of what SWEs should be doing these days... Code is easy to produce now. We should be handing over the creation and shipping of code to the relevant teams.
Maybe there will be more complexity overall, but it will be constrained via well designed microservices and distributed across the organisation.
I just did the LoC measure that isn't "official" but actually closer to how AI would read it: I.e. simply actual lines. Never mind "context window".
Just in the actual source folder there are 1.9 million lines. That's not tokens. Lines. Not even LoC as per some official canonicalized definition. Various languages. ~15 year old code base.
We work with Claude just fine.
For variable measures of "fine" depending on how much devs just trust whatever Claude spits out ;)
Current project has over 60,000 LOC, I find using an antagonistic agent code and architecture reviews are really handy for keeping things grounded.
Is there ANY field where good engineers don't understand the basics? Can an electrical engineer do any work without understanding Volts and Amperes?
This is not a silly question, because it's kind of silly:
Has an AI ever actually gone through all of the steps necessary to become a chess master? Go to competitions, raise in the ranks, take a plane to the masters, organize support and all the real life details necessary? Playing chess is just the small formal part here, and all of the obvious things humans obviously do, well...
Chess is a perfect information game with objective, easily measurable outcomes.
Show me a software business that looks like that and I'll agree with your hypothesis.
Otherwise I don't believe your metaphor proves much of anything.
I disagree, currently a human guiding an AI agent is far better (more productive for less money) than an AI agent with very loose non-technical guidance and The Ralph Wiggum Loop.
I don't know how much longer humans have, but I don't think they can be cut out of the loop entirely just yet.
Please never compare perfect information games (https://en.wikipedia.org/wiki/Perfect_information) with actual reality, which is probably many orders of magnitude more complex.
Kind of similar story with math (pure) and other sciences (impure): https://xkcd.com/435/
OpenAI & co keep presenting their achievements at maths competitions as their path to AGI/ASI/singularity. But if you look at that XKCD graph, the way the joke reads is in terms of arrogance (scientists working on purer fields are more arrogant), but the actual science complexity work the exactly the other way around. Impure sciences are orders of magnitude more complex.
Implementing a sorting function = coding (relatively pure).
Creating an entire commercial software product/service with high performance and other professional-level characteristics = software engineering (wildly impure; after 60+ years we can't even agree on the best programming language to use).
I don't think people have identified that as an LLMism, but the way it feels out of place I wouldn't be surprised if someone did. Being a punchy two word sentence doesn't help here.
Edit- it being weird and out of place is probably right because I probably used it wrong. I probably shouldve said this lands for me. That sounds more normal
Edit2 I am not a bot trying to improve to sound more human. I promise
Furthermore the AI companies are spending gobs of money and compute to address this—harness development, orchestration/agent coordination, computer use are all designed to lower the skill level required to get the best results out of a given model.
Soon the models themselves will be more persistent, more observant, and more skilled than any of us at driving models to obtain the "right" solution in a given SWE context.
Where they still fall massively short, even compared to a straight out of college junior developer, is figuring out what the correct question to ask is.
This has led to situations like Fable 5 spending six days debugging a deadlock because it didn't think to add any observability to detect when the job stalls, and would wait until the job timed out each time. Any junior developer would have fixed it in a few mins, but I left Fable running out of curiosity to see how long it'd take.
If anything I feel I am understating the capability of current models here. What is stopping the next generation of models from closing the feedback loop further?
An explicit query returns a specific answer, but means nothing about whether it can use that knowledge to create better software.
"Median" is kinda hard to judge given how rapidly the number of software engineers has been growing, how imprecise our within-industry skill assessments are, and also how fast everything within software was shifting even before agentic coding. Two examples: (1) an old estimate I heard was that 50% of the skills we would put on our CVs would be obsolete every few years, but we'd never know which ones; (2) The reason for the creation of FizzBuzz as an interview question (far too many people are *that* bad).
For the LLMs themselves: the METR time horizons estimates are based on an expert developer's wall-clock times, and the graph on their website only goes as far as Mythos preview which is kinda about this level. Mythos preview shows only 80% success at tasks that would take a human 3h6m on average (95% confidence interval 1h37m-6h39m), which is evidence that yes, there is plenty that the models can't do. I naturally break down larger tasks to smaller ones, the agentic coding models I've used also attempt to do this, but seem to be worse at it than I am.
> It strikes me that models already have superhuman software architecture capabilities, they are simply underelicited.
While they can do better when elicited correctly (and the same is true for humans), they are too "lazy" by themselves to attempt self-elicitation.
The second part, just like the first, is also true for humans; but LLMs are so much cheaper than humans that this is a disappointment for the LLM to stop early and say "[Thought for 35m 24s] Here's your MVP, I've not done subtasks 35 onwards", even though it's an advantage for the humans to say "[Invoice for a 2 week sprint with a team of 5 developers and their manager for a total cost of $23,000] Here's your MVP, we can discuss which of subtasks 35 onwards you want in the next sprint".
I'm not sure where I'd rate LLMs on a strict 0-10 scale from "hack" to "formal methods followed by end-to-end testing", but the examples given by the research companies themselves suggest that even they are only at the "this is starting to get interesting" level for both architecture and their own idea of what "I have now completed this to a good quality" looks like, not at the "actually good" level for complete big projects.
> Furthermore the AI companies are spending gobs of money and compute to address this—harness development, orchestration/agent coordination, computer use are all designed to lower the skill level required to get the best results out of a given model.
The observation is that money and compute spent are not strongly correlated with the general quality of the models. If the correlation was strong then Meta would be near the top, while the multitude of open-weight Chinese models that keep popping to the top of leaderboards every couple of months would be jokes. Grok would have been better from the start and not suddenly gotten good around when they bought up the much smaller and lower-budget Cursor.
https://llm-stats.com/models/muse-spark-1.1
https://llm-stats.com/models/gpt-5.6-sol
https://llm-stats.com/models/kimi-k3
> Soon the models themselves will be more persistent, more observant, and more skilled than any of us at driving models to obtain the "right" solution in a given SWE context.
My experience has been that predictions of "soon" fare about as poorly as predictions of "never" in the context of AI. Regarding the failures of "soon" predictions, I do not know the degree to which this is company leadership BSing how good their stuff is, vs. Goodhart's law where they're fooled by the difference between their metrics and reality.
At the same size I only heard of AI generated projects reimplementing compilers and browser engines which come with amazing tests written by hand, but even these projects generally don't become reliable enough to use.
That 1.9m code base, within the same time frame as the 160k was created? <10k by the same measure.
10k I can comfortably throw at a local agent on my system (see recent posts in my history). Whether I can keep it that small is another question I suppose lol!
write a comment about how to use ChatGPT to write a game like you're a barely literate reddit user, short answer, maximum incoherency
Not perfect, gptzero.me still knew it was AI generated (tayo42 is human by the same measure), but if for some reason someone was using an LLM prompted with that pattern, I suspect it would fool me in a Turing test unless I found the keyword to force the agent to change the role it was playing.
Conveying the errors typical in a Raise The Colours group on Facebook, that's the hard stuff.
Sometimes they do. From Gemini 3, some AppleScript:
if something then
if something then
if something then
stuff
end if
end if
</if>
After about 10 turns the problem went away.
I do very little actual reading of code anymore. Now I am more of a results merchant.