This article reads like complete AI, this is sad because it raises a point but I wish the author actually wrote the f**ing article. Just share us the prompt next time

     A perfectly optimized system that produces incompetent operators is not optimized at all. It has simply moved its failure mode somewhere the spreadsheet can’t see it.

I think a problem is that especially with the culture of job hopping a lot and high turnover you can't be sure that you're not just training staff that'll go elsewhere. It's like an industry wide prisoners dilemna.

First, they made CI/CD pipelines to automate the builds and deployment - and I said nothing because I didn't want to watch the scripts run. It's only running my code anyway, and more automation is good. Less for me to do!

Then, they integrated AI into our IDE - and I said nothing because the guardrails kept it in check. It's only writing code following my instructions, and more automation is good. Less for me to do!

Then, they requested source control access so the business owners could skip all the feature planning back-and-forth and just chat with an AI agent until the feature looks how they want it - and I said nothing because they're not merging into master. I'm still in charge of reviewing code, and I have an AI agent helping me review code using some rules and invariants I've handed it, and more automation is good. Less for me to do!

Now, they are requesting an always-on AI agent reviews and merges their code into master with 0 human interaction because their PRs which contain 300K lines changed each have started to pile up over the last 3 days. And I have to just do it because boss man pays my bills. But hey, more automation is good. Less for me to do.... until everything breaks and I have 1 hour to find why the million lines of code merged this week don't work anymore.

----

The story makes it sound like I didn't push back. I did. But after being forced to ignore a full career's worth of red flags and best practices I'm now just whoring my skills out to see how deep this AI-fully-replacing-devs rabbit hole goes. Besides, when else am I going to have crazy money to just throw at a new server and set it up to replace the SDLC end-to-end with a rich man's AI? And there's still the fun technical challenge of "can I make this system secure enough that even when I automate myself out of a job I couldn't revenge hack it?"

(And yes, it's telling that the author has clearly used AI to generate the argument, because not only does it weaken their argument, I think they miss the point slightly because of their reliance on it.)

They focus on safety-critical systems in aviation, which is narrow part of all industries, when the bigger issue is organisational resiliency and reproduction. The skills that allow an organisation to weather change and disruptive market conditions are not tied up in proprietary information, but are captured as unwritten social rules and context that isn't passed on when labour is outsourced, either off-shore or via computerisation.

The problem is particularly acute in software development, because software engineers not only make the system that encodes proprietary knowledge and organisation advantage, they maintain it, and without a continual replacement of labour that understands the system and the reasons for the system's behaviour, the ability to extend it or fix it is lost.

This problem predates AI, and even software, where organisations die because they outsourced too much of their critical internal function. It eventually manifests at an industry level, where the critical knowledge to do a certain thing is lost (see, for example, the ability to produce new IMAX projectors as a microcosm, but also the loss of machine tooling skills, etc.) - at the heart of why some countries subsidise inefficient industries because the firms within them have no incentive to preserve themselves but their skills are invaluable to preserving national autonomy.

This slop is embarrassing

When the skilled workers start to run out, either they find new workers to train, or they replace the thing that needed skilled workers (because the thing is probably quite old and in need of replacing anyway). This happens to virtually all kinds of human industry. The world doesn't fall apart when very old skills start to disappear. We're humans. We adapt. We can't even stop ourselves from doing it, it's in our DNA.

Worst case, what happens if everyone forgets how to write Python? 1) You can pay someone to sit down and read books on Python, 2) you can replace the Python with Go. That second one is actually the better choice since the AI can already code in all these languages. But what happens if all the books are gone and the AI can't code anymore? You make a new language. How will you know how to make a language? Either you ask the AI how, or you figure it out the same way humans did the first time. Or some other solution, of which there are dozens.

The point is, there is no skill cliff we're going to fall off of. It's an irrational fear. If the people who have this fear would stop and think about it for 20 minutes, they could come up with a long list of ways to respond, other than "keep an army of Juniors around doing things by hand so nobody ever forgets anything".

(the plane example was stupid, btw. almost nothing in the world is as difficult to recover from failure as a 400-ton metal cylinder going 400mph at 30,000 feet in the air. most of the time it's fine to just call a repair guy. for the cases where you can't, invest in better process and better certification/recertification)