I'm noticing some of the concern play out regarding AI weakening the capabilities of software people.

I gave the team an exact solution on a silver platter and they still failed to identify how to go about it after 3 days slamming it into Claude. The resolution is literally 1 line of code that could be arrived at in about 30 minutes of patient, old school troubleshooting.

I think what's happening is the AI system draws poorly aligned and led engineers into this ego inflation feedback loop where they are completely detached from reality because these tools can simulate a better one.

We do not yet have the next generation systems that will manage AI creation and maintenance of systems.

Humans have been making spaghetti code systems and maintaining them poorly for years. Best practices developed… eventually. But certainly not in the 70s and 80s. Spaghetti was the norm for quite some time.

The development paradigm has changed. Forever. You can’t expect yesterday’s tools for managing software development to succeed at this point. We’re still roughly on year one of this transformation.

The new bottlenecks are creating and enforcing boundaries in the code, identifying level of risk within a boundary, subjecting high risk areas to more intense human review and architecture reworks, and so so much more manual testing.

We need a new language for high level development that focuses on architectural constraints. We need analysis software that helps draw boundaries, identify what needs extended human attention, and helps us map and understand a rapidly developing code base. We need to standardize on the use of frameworks and languages like we have on assembly variants.

I’ll bring up a hacker news trope here. NaughtyDog’s GOAL was an amazing system that was a product of an underdeveloped ecosystem for game development at the time. They used lisp and assembly because you still had to write assembly at the time.

We are in a very similar time. The ecosystem is underdeveloped. We should be looking for new languages and tools to manage this.

CNC machines used to run from punch cards and then hand written NC code. Now we have advanced CAM software. (Which innovative people are actually running with LLMs!)

The more I use it, the more I have to rely on it to make changes/ fix things in the same system. In the end, I come out feeling empty; no intuitive knowledge of the system "I" built or fixed.

Code review is important but it does not replace the mental model I am able to build when I do all the steps of software development manually without AI.

Luckily, there are ways to mitigate this and essentially translate those human intuition of how the codebase “should” be into guardrails for the agents. But without that, your setting your sails in a stochastic sea where each wave looks nothing like the last.

I like the plane example from the article,but I think in reality it will be like code. 1.5 years ago engineers would routinely say that they still write code by hand here or there to keep their skills sharp, and that's just not something you hear much if at all.

If an SRE is faced with a situation an AI can't solve, then said SRE will use the AI systems to triage further, point it to different places and so on.

This works for SREs with pre-AI experience and intuition, possibly less so with new recruits coming in post-AI. I don't know what the solution to this is, maybe practice drills is it, but I have a hunch the entire field will be subsumed, same as many other engineering fields.

There is only so much need for taste and judgement, before even that has been incorporated into the models.

I work with programmers and it's not uncommon that they can remember with shocking detail about code they've written in the past.

Someone might mention an issue that has cropped up and they'll stare off into space for a few moments and actually remember where that issue stems from in the code, because they remember writing it like 8 months ago.

This skill will be lost when AI is generating all code, we'll be stuck in a perpetual loop of having AI keep track of the state of the code in order for AI to extend and maintain it.

They were asked to "just use AI" and understand the component, triage the issue, build a fix etc. The engineer was forced to choose between accepting a potentially mediocre fix AI has suggested or risk being coming across as an incompetent resource who doesn't know how to leverage AI.

You can guess what the engineer chose. The fix wasn't bad but it was suboptimal for some edge cases. We had to later revise it. Have enough of these situations, engineers will eventually definitely give up understanding the system in detail.

Audit trails, logs, and tight data governance where things can only be accessed with proper roles is the only possible solution.

If an RPA team ever gets direct access to a production database in your company, look for a new job.

One day, those may become bigger as they are forgotten, causing havoc. The standard root-cause-analysis depending on systems having certain retention period, which may be expired at that time.

It is important to get real hold of one's systems from end-to-end aspect, which holds true for both AI and human operators...

Depending on what your goals as the author are, you may or may not want that.

Being able to scroll it out of view might be enough to achieve the aesthetics goal, and the goal of people actually listening to you.

I agree with premise of thr blog. The question i have been asking internal does knowing your system really matter if you can recreate it in minutes.

We recently had a situation, where in with our internal platform and claude we recreated everything in minutes.

Management in the end cares about the outcome and not how the meat is made.

An AI can handle routine incidents and then present learning cases from that routine work for training, because the skill in SRE is not the mechanical log grepping, grafana dashboard browsing etc but forming the hypothesis. AI incident reports can create training cases that are a much better training for hypothesis forming and testing than the work itself can.

Even before layoffs many teams just maintained things org has long lost coherent knowledge of

After layoffs and typical org knowledge churn - you can either rewrite it (but how? Product team responsible for original implement requirements is long gone too) or recoup (reverse document) some of that lost knowledge with AI and actually learn

Also, that paper "The Ironies of Automation" is one that everyone should read. It's fairly short.

There is a related problem in terms of these situations where the computer system is handing off to the human. It's called "the bumpy transfer of control." Very fascinating concept.

Reminds me of the move from on prem to cloud. Linux sysadmins were killed and replaced by aws focused devops.

That doesn’t mean the proposed solution is always right, but it is absolutely landing on the root issue faster than most humans would be able to, even pre-AI. Just because it can remember and run through a bunch of commands more quickly that I can.

There are lots of incidents where the symptom doesn’t always clearly point to the issue, so having something that can fairly exhaustively check a lot of different things very quickly is pretty useful!

But I at least partly agree, I think the more complicated and obtuse a system is, the harder it is for AI to do this. But we’ve invested time in making systems much more straightforward to understand and operate from one place (Kubernetes in general), and AI thrives on that.

And yeah, it means your debugging skills wane a bit because, yeah, if the bot can diagnose the issue in 5 minutes, it’d be irresponsible to not use it.

And I’m not really happy about it, and personally I’ve always been able to figure out a tricky bug given enough time. I don’t want to loose that skill. But everyone is under a lot of time pressure these days.

I disagree with this. Whatever the AI produces must be embodied in some kind of information system. The moment the output is on disk, it's fish in a barrel for any competent operator.

I've worked in environments that are beyond the pale with regard to complexity. It will take AI another 10 years to product something as complicated and coherent as a semiconductor manufacturing operating system, which is clearly feasible for humans to manage today.