First, I disagree that good code is required in any sense. We have decades of experience proving that bad code can be wildly successful.

Second, has the author not seen the METR plot? We went from: LLMs can write a function to agents can write working compilers in less than a year. Anyone who thinks AGI is far away deserves to be blindsided.

When you let an LLM author code, it takes ownership of that code (in the engineering sense).

When you're done spending millions on tokens, years of development, prompt fine tuning, model fine tuning, and made the AI vendor the fattest wad of cash ever seen, you know what the vendor will do?

You have to migration path. Your Codex prompts don't work the same in Claude. All the prompts you developed and saved in commits, all the (probably proprietary) memory the AI vendor saved in their servers to make the AI lock you in even more, all of it is worthless without the vendor.

You are inventing "ah heck, we need to pay the consultant another 300 bucks an hour to take a look at this, because nobody else owns this code", but supercharged.

You're locking yourself in, to a single vendor, to such a degree that they can just hold your code hostage.

Now sure, OpenAI would NEVER do this, because they're all just doing good for humanity. Sure. What if they go out of business? Or discontinue the model that works for you, and the new ones just don't quite respond the same to your company's well established workflows?

You're all being fooled by emergent behavior, which acts like intelligence, but really doesn't fool people who are familiar with how to write code.

I'm sorry, I'm not sure how to say this without sounding elitist, but the goal post has not moved since GPT 3. These tools, autonomously, produce code that only fools the clueless. I don't know how else to put it and I'm getting really tired of this argument that "look, company with billions and some of the buggiest shittest software is using AI to write it". No shit they are.

So what's going to happen? We will see the same divide we've seen with JavaScript. It's new, then everyone says it's what everyone must use. C++ developers no longer needed--its all JS now. No need for native UIs, it's all JS now. If you're not learning the latest web tech, you'll miss out and fall behind. If you're studying for anything but web, you'll be out of a job in 5 years. And now, a couple decades later, we are still waiting for it.

"In order to make machines significantly easier to use, it has been proposed (to try) to design machines that we could instruct in our native tongues. this would, admittedly, make the machines much more complicated, but, it was argued, by letting the machine carry a larger share of the burden, life would become easier for us. It sounds sensible provided you blame the obligation to use a formal symbolism as the source of your difficulties. But is the argument valid? I doubt."

I'm sure it will be possible, but it may well be very expensive. If it is, why would anyone spend the resources?

AI evolution will certainly follow the money, which is not necessarily the same as the path to AGI.

1 - “It seems like 99% of society has agreed that code is dead. …It's the same as thinking storytelling is dead at the invention of the printing press. No you dummies, code is just getting started. AI is going to be such a boon for coding.“

2 - Another one comparing writing and coding, and explaining how Code is both a means and an end to manage complexity:

“we're confused because we (incorrectly) think that code is only for the software it produces. It's only partly about that. The code itself is also a centrally important artifact… I think this is a lot clearer if you make an analogy to writing. Isn't it fucking telling that nobody is talking about "vibe writing"?”

That's what happens when you hand everything to a machine without understanding the problem yourself.

AI can give you correct answers all day long, but if you don't understand what you're building, you'll end up just like the people of Magrathea, staring at 42 and wondering what to do with it.

True understanding is indistinguishable from doing.

Maybe but all technologies have limits.

It's irrational to believe any single technology can be improved forever.

I know of quite a lot of business people who kind of frolicking over the idea that the former behemoth got humbled so massively. From eating the world to unemployed in no time.

This delusion itself is telling and perpetuates the clinging to the sinking ship that is still the Elephant in the Room.

If something as complex or even complicated as app development including SDLCs etc. could simply be prompted now, then AI will eat anything less complicated alive.

So people better start considering the implications and ramifications of their statements. Otherwise we are all doomed and part of a darwinian system that will weed out the unnecessary parts of the system or we acknowledge the fact that a traditional profession such as app development is fundamentally changing.

This is something that happened before and constantly does. Otherwise we would not use DSL or Java.

But the fundamentals still work and therefore you need abstractions.

We may expect code to be killed off in AI's troublesome teen years.

Hope this does not happen to code :)

Maybe in the future us olds will get more credit when apps fall over and the higher ups realize they actually need a high-powered cleaner/fixer, like the Wolf in Pulp Fiction.

B) They are shit for frontend work. This is the big one, and maybe we will someday, but we really haven't bridged the gap between an LLM seeing code and matching it to something on screen. People are well aware that agents need tight feedback loops to really be effective, which is more doable for server side code where you can setup a good test apparatus (still a pain in the ass for other reasons). But the thing a developer does most often is instantly see what their code changes did to a project visually. This is a huge gap with LLMs. We are at the level where an LLM can check that the site loads to begin with. That's it. We can't trust it to make functioning UI, and we sure as hell can't trust it to compare code against UI unless we ask it something very specific. Watch the LLM clocks sometime [1]. I was shocked to see even 5.2 codex making complete nonsense every few minutes.

C) My coworkers have narrow experience in the field we work in, which means they are frequently zooming past things that a general approximation of intelligence like Claude gets tripped up on. ChatGPT is trained on the entire corpus of humanity, yet doesn't understand there is such a thing as bad language servers. That a linting error is not inherently problematic and you don't need to spend $5 in tokens investigating it. This is something a junior developer understands. Why do I need to constantly watch and stop my agent and re-explain things we learned in the first week of programming?

D) Because LLMs are an approximation of correct solutions, god help you if you work in spaces where your syntax looks similar to other frameworks. I work in a lot of legacy apps that are framework spin-offs of other framework spin-offs. Stuff that's half documented and poorly made. There is no testing for these projects and no good IDE support. So if something is an offshoot of OctoberCMS, ChatGPT is just going to make up a bunch of methods from OctoberCMS even if they are completely invalid in this project. Agents are completely useless in projects like these.

E) Local models aren't there yet, and the fact that the entire industry is cool with letting their skills atrophy so hard that they are completely dependent on the livelihoods and structures of a few companies they hadn't even heard of 5 years ago is deeply concerning and unprofessional. I personally wouldn't hire someone so short sighted. Believing in LLMs taking over all coding work is a fundamental disbelief in Humanity, and I'm just not behind it. The most amazing thing about development is how much knowledge share we have and how cooperative we are. A future where no one knows how to do anything anymore is the future where tech truly goes off the rails.

[1] https://clocks.brianmoore.com/

My experience is it hits both new-feature velocity and stability (or the balance between those two) really early, but lots of managers don't realize that this feature that's taking literal months could have been an afternoon with better choices earlier on (because they're not in a position to recognize those kinds of things). For that matter, a lot of (greener) developers probably don't recognize when the thing that's a whole-ass project for them could have been toggling a feature flag and setting a couple config entries in the correct daemon, with better architecture, because... they don't even know what sort of existing bulletproof daemon ought to be handling this thing that somehow, horrifically, ended up in their application layer.

So the blame never gets placed where it belongs, and the true cost of half-assed initial versions is never accounted for, nor is it generally appreciated just how soon the bill comes due (it's practically instantly, in many cases).

That might be valid, if LLMs stopped improving today.

However AI systems in 2026-ε were utterly inadequate at coding

And AI systems in 2026+ε might not have the present limitations

Well, of course. Despite people applying the label of AI to them, LLMs don't have a shred of intelligence. That is inherent to how they work. They don't understand, only synthesize from the data they were trained on.

And yet the AI probably did better than 99% of human devs would have done in a fraction of the time.

In theory, we are just one good innovation away from changing this. In reality, it's probably still some years away, but we are not in a situation where have to seriously speculate with this possibility.

> And this is why humans will be needed to advance the state of the art.

But we only need a minority for innovations, progress and control. The bulk of IT is boring repetitive slop, lacking any innovation and just following patterns. The endgame will still result in probably 99% of humans being useless for the machinery. And this is not really new. In any industry, the majority of workers are just average, without any real influence on their industries progress, and just following conventional wisdom to make some bucks for surviving the next day.

The rate of change is accelerating. I worry we don't have much time left unless we get serious about merging with machines.

We don't necessarily need a Chris Lattner to make a compiler now.

End of the day Chris Lattner is a single individual, not a magic being. A single individual posting submarine ads for his cleverness in the knowledge work subfield of language compilers. Of course he is going to drag the competition.

Languages are abstraction over memory addresses to provide something friendlier for human consumption. It's a field that's decades old and repeats itself constantly being it revolves around the same old; development of a compression technique to deduplicate and transpile to machine code the languages more verbose syntax.

Building a compiler is itself just programming. None of this is truly novel nor has it been since the 60s-70s. All that's changing is the user interface; the syntax.

Intelligence gives rise to our language capacity. The languages themselves are merely visual art that fits the preferences of the language creator. They arbitrarily decided nesting the dolls their way makes the most sense.

Currently have agents iterating on "prompt to binary". Reversing a headless Debian system into a model and optimizing to output tailored images. Opcodes, power use in system all tucked into a model to spit back out just the functions needed to achieve the electromagnetic geometry desired[1]

[1] https://iopscience.iop.org/article/10.1088/1742-6596/2987/1/...

  for host in vps_hosts; do
     ssh $host < commands.sh > errors.log 2&>1 &
  done

Relationships in the physical world are sparse, metastable graphs with non-linear dynamics at every resolution. And then we measure these dynamics using sparse, irregular sampling with a high noise floor. It is just about the worst possible data model for conventional AI stacks at a theoretical level.

[1] https://en.wikipedia.org/wiki/Softmax_function

When the TPU does it, is the TPU thinking? Where does the critical thinking take place in the endless pages of matrix math that eventually evaluates into the same token output as the TPU?

The original post in Chinese was handwritten

https://blog.est.im/2026/stderr-03

the English translation was compiled by gemini3.

Yes, we'd love to visit!

But more importantly, as a coding agent, it follows instructions much better. I've frequently had Claude go off and do things I've explicitly told it not to do, or write too much code that did wrong things, and it's more work to corral it than I want to spend.

Codex will follow instructions better. Currently, it writes code that I find a few notches above Claude, though I'm working with C# and SQL so YMMV; Claude is terrible at coming up with decent schema. When your instructions do leave some leeway, I find the "judgment" of Codex to be better than Claude. And one little thing I like a lot is that it can look at adjacent code in your project so it can try to write idiomatically for your project/team. I haven't seen Claude exhibit this behavior and it writes very middle-of-the-road in terms of style and behavior.

But when I use them I use them in a very targeted fashion. If I ask them to find and fix a bug, it's going to have as much or more detail as a full bug report in my own ticketing system. If it's new code, it comes with a very detailed and long spec for what is needed, what is explicitly not needed, the scope, the constraints, what output is expected, etc., like it's a wiki page or epic for another real developer to work from. I don't do vague prompts or "agentic" workflow stuff.

It's like Stephen King saying an AI generated novel isn't as good as his. Fine, but most of have much lesser ambitions than topping the work of the most successful people in the field.

I have forgotten more about chip and electrical engineering than he will ever know.

Manager people or managing a hyper-knowledgeable intern (LLM). If you know what you need, actually want what you want (super difficult), and have the ability to provide context to someone else… management has always been easier for you than others.

I find one of the more interesting things about the current “AI debate” is that many programmers are autistic or at least close one side of an empathetic spectrum that they’ve always had trouble communicating what is needed for a task and why. So it’s hard for me to take the opinions going around.

> That's irrelevant.

My comment was relevant, if a bit tangential.

Edit: I also want to say that our attitude toward machine vs. human intelligence does matter today because we’re going to kneecap ourselves if we incorrectly believe there is something special about humans. It will stop us from closing that gap.

Today, yes. I assume in the future it will be integrated differently, maybe we'll have JIT fine-tuning. This is where the innovation for the foundation model providers will come in -- figuring out how to quickly add new knowledge to the model.

Or maybe we'll have lots of small fine tuned models. But the point is, we have ways today to "teach" models about new things. Those ways will get better. Just like we have ways to teach humans new things, and we get better at that too.

A human seeing a new programming language still has to apply previous knowledge of other programming languages to the problem before they can really understand it. We're making LLMs do the same thing.

A while ago I spent some time debugging a superfluous redirect and the reason was that the library would always kick off with a "not authenticated" when it didn't find stored tokens, even if it was redirecting back after successful log in (as the tokens weren't stored yet).

It's not a SAT solver (yet) and will have trouble to precisely handle arbitrarily large problems. So you have to lead it a bit, sometimes.

To your point, yeah, I mostly don't want AI to be innovative unless I'm asking for it to be. In fact, I spend much more time asking it "is that a conventional/idiomatic choice?" (usually when I'm working on a platform I'm not super experienced with) than I do saying "hey, be more innovative."

Further model improvements would be a bonus, but it's not required for us to get much further.

  > Writing React code is very tedious.

i feel like llms are abstracting away that tedium sometimes yes, but i feel its probably because the languages and frameworks we use aren't hitting the right abstractions and are too low level for what we are trying to do... idk just a thought

You are arguing against the internet, motor cars and electricity.

It's like 1998, and you're saying: "I'm sure it's possible there are cases where the internet can be useful. I'm not sure it is though"

On 'hackernews' of all places.

It's pretty wild to see that, and I think it says something about what hn has become (or maybe always was?).

The point is - there are so many combinations and permutations of reality, that AI can easily create synthetically novel outcomes by exploring those options.

It's just wrong to suggest that 'it was all in some textbook'.

"here’s just no reason to believe that it is anything more than it is "

It's almost ridiculous at face value, given that millions of people are using it for more than 'helping to write react apps every day'.

It's far more likely that you've come to this conclusion because you're simply not using the tools creatively, or trying to elicit 'synthetic creativity' out of the AI, because it's frankly not that hard, and the kinds of work that it does goes well beyond 'automation'.

This is not an argument, it's the lived experience of large swaths of individuals.

LLMs remove the time problem (to an extent) and have more problems around understanding the constraints imposed by the framework. The trade-off is less worth it now.

I have stopped using frameworks completely when writing systems with an LLM. I always tell it to use the base language with as few dependencies as possible.

I disagree with this premise. A computer approximates a Turing Machine, which puts it far above a brick.

If we're going to be reductionist we can just call humans "meat sacks" and flip the question around entirely.

Now that's extrapolation of the sort that, as you point out elsewhere, no LLM can perform.

At least, not one without serious bugs.

Comercial fusion is still decades away.

Solar and lithium batteries are getting cheaper every year but unfortunately still lots of political issues in many countries.

> Wait, which theory is that?

The Church-Turing-Deutsch Principle. (Which isn’t a theory in the empirical sense, but somewhat more speculative.)

> Or are we suggesting that you can build a computer out of whatever physical process you want to model?

Well, you obviously can do that. Whether that computer is Turing equivalent, more limited, or potentially a hypercomputer is...well, Church-Turing-Deutsch says the last is always false, but good luck proving it.

Robotics has passed the point of superhuman performance for any given task. Software has passed the point of superhuman performance for any given task.

Regardless of the particular technique or embodiment, the constraints aren't "is it possible in principle" but "is it too expensive" and "is this allowed by the pertinent principles and regulations and laws"

We don't have AGI that learns and adapts in real time like humans. We do have incredibly powerful algorithms that can learn from whatever data we throw at them, but many domains where it's impractical, ruinously expensive, illegal, or otherwise not possible to use AI for some other good reasons.

The few islands left to humanity are not fundamental barriers. We haven't solved intelligence, or achieved RSI or ASI or AGI yet; those were never the important thresholds.

AI has always been a question about good enough, and it looks like we've gone solidly past the good enough line into "we can probably automate everything" even if we don't solve the big problems over 5 or 10 years or beyond. I think it's very unlikely we don't solve intelligence by 2030, but even if AI stalls out where it's at right now, and all we get is the incremental improvements and engineering optimizations on current SOTA, we have enough to automate anything humans do at levels exceeding human capabilities.

What AGI and ASI do is make humans economically obsolete. Good enough AI means there might be some places where humans are needed for generalization and adaptability until the exhaustive tedious work gets done for a particular application that enables a robot or software system to be competent enough to handle the work.

https://www.invene.com/blog/limiting-developer-turnover has some data, that aligns with my own experience putting the average at 2 years.

I have been doing this a long time: my longest running piece of code was 20 years. My current is 10. Most of my code is long dead and replaced because businesses evolve, close, move on. A lot of my code was NEVER ment to be permanent. It solved a problem in a moment, it accomplished a task, fit for purpose and disposable (and riddled with cursing, manual loops and goofy exceptions just to get the job done).

Meanwhile I have seen a LOT of god awful code written by humans. Business running on things that are SO BAD that I still have shell shock that they ever worked.

AI is just a tool. It's going from hammers to nail guns. The people involved are still the ones who are ultimately accountable.

> it would take a few years at least

I think a good example are calculations or counting letters: it's trivial to write turing machines doing that correctly, so you could create neural networks, that do just that. From LLM we know that they are bad at those tasks.

It also doesn't say anything about finding the desired function, rather than a different function which approximates it closely on some compact set but diverges from it outside that set. That's the trouble with extrapolation: you don't know how to compute the function you're looking for because you don't know anything about its behaviour outside of your sample.

As an example, I would like to ask how to parse balanced brackets grammar (S ::= B <EOS>; B ::= | BB | (B) | [B] | {B};) with that Turing complete recurrent network and how it will deal with precision loss for relatively short inputs.

Paper also does not address training (i.e., automatic search of the processors' equations given inputs and outputs).

Mathematically possible != actually possible.

> unstated assumption that technological progress towards human-like intelligence is in principle possible. In reality, we do not know.

For me this isn’t an assumption, it’s a corollary that follows from the Church-Turing thesis.

Reply due to rate limiting:

K&R didn't know about CI/CD, but everything else you mention has either existed for over 30 years or is too trivial to argue about.

Conversely, if you took Claude Code or similar tools back to 1996, they would grab a crucifix and scream for an exorcist.

LLMs are really good at doing that. Arguably better than humans at RTFM and then applying what's there.

Citation needed.

Explaining it as something like "realizable instantiation of physical computation occurring in the universe mapping to an ultra-sparse, discrete point cloud embedded in the Euclidean parameter space of all computable functions" could definitely be more precise, but you're either going to need a topology like a landscape or a bumpy sphere to visualize it, and then you're going to need to spend more time showing the effects of things like scaling laws, available compute, where the known boundaries of human intelligence lie, and so on, and so forth, and by then you've lost everyone, probably even the ML professor.

It's a good enough metaphor that maps to a real thing.

Right. What you're saying is that barely anyone is doing truly novel work. 100% agree.

"LLMs can only emit things they've been trained on" is wholly obsolete.

This is broadly true, but not comparable when you get into any detail. The mistakes current frontier models make are more frequent, more confident, less predictable, and much less consistent than mistakes from any human I'd work with.

IME, all of the QA measures you mention are more difficult and less reliable than understanding things properly and writing correct code from the beginning. For critical production systems, mediocre code has significant negative value to me compared to a fresh start.

There are plenty of net-positive uses for AI. Throwaway prototyping, certain boilerplate migration tasks, or anything that you can easily add automated deterministic checks for that fully covers all of the behavior you care about. Most production systems are complicated enough that those QA techniques are insufficient to determine the code has the properties you need.

People have been doing thigs millenia before they understood them. Did primitive people understood the mechanism behind which certain medicinal plants worked in the body, or just saw that when they e.g. boil them and consume them they have a certain effect?

We've only had the tech to be able to research this in some technical depth for a few decades (both scale of computation and genetics / imaging techniques).

But sure, instantiating these capabilities in hardware and software are beyond our current abilities. It seems likely that it is possible though, even if we don’t know how to do it yet.

And of course, if you don't limit yourself to "advancing the state of the art at the far frontiers of human knowledge" but allow for ordinary people to make everyday contributions in their daily lives, you get even more. Sure, much of this knowledge may not be widespread (it may be locked up within private institutions) but its impact can still be felt throughout the economy.

That feels overly optimistic. LLMs seems on track to automate out basically any "email job" or "spreadsheet job," in which case we'll be looking at higher unemployment numbers than the great depression for at least some period of time. Combine with increased automation...

There are a LOT of people in the world and already a not insignificant portion can't find work despite wanting to. Seems the most likely thing is that the value of most labor is reduced to pennies.

> Turing completeness not necessary for creativity--not by a long shot.

This is by far a more extreme claim than the others in this thread. A system that is not even Turing complete is extremely limited. It's near impossible to construct a system with the ability to loop and branch that isn't Turing complete, for example.

>(In fact, humans are not Turing complete.)

Humans are at least trivially Turing complete - to be Turing complete, all we need to be able to do is to read and write a tape or simulation of one, and use a lookup table with 6 entries (for the proven minimal (2,3) Turing machine) to choose which steps to follow.

Maybe you mean to suggest we exceed it. There is no evidence we can.

> all we need to be able to do is to read and write a tape or simulation of one

An infinite tape. And to be Turing complete we must "simulate" that tape--the tape head is not Turing complete, the whole UTM is.

> A system that is not even Turing complete is extremely limited.

PDAs are not "extremely limited", and we are more limited than PDAs because of our very finite nature.

  > This is meaningless.

  [1] https://en.wikipedia.org/wiki/Constructivism_(philosophy_of_mathematics)#Constructive_mathematics

You just compute the brains of a bunch of immortal mathematics. At which point it's "very difficult and expensive function to evaluate with absurdly large boundary conditions."

Where's the proof we don't do exactly this? The mind as a prediction engine is one of the handful most accepted theories.

Of course it does, but only after the fact. You don't have any experience of the result of the experiment before you perform it.

Sure, they can't have apples fall on their heads like Newton had but they can totally observe the apple falling on someones head in a video

I have strong doubts that LLMs have any understanding whatsoever of what's happening in images (let alone videos). The claim (I've sometimes heard) that they possess a world model and are able to interpret an image according to that model is an extremely strong one, that's strongly contradicted by the fact that they: a) continue to hallucinate in pretty glaring ways, and b) continue to mis-identify doctored (adversarial) images that no human would mis-identify (because they don't drastically alter the subject).

Btw, it looks like there is a growing body of research evaluating exactly this. I found this nice overview with even some benchmarks specifically for scientific innovation: https://github.com/HKUST-KnowComp/Awesome-LLM-Scientific-Dis...