It's not clear to me why this is true. If LLMs are writing code, why are developers simply not orchestrating the completion of more features instead of moving up the stack to do product development work? Is there some implication that the existence of LLMs also enables developers to run user studies, evaluate business metrics and decide on strategy?

Additionally, if PMs can use LLMs to increase velocity in their work why not focus on all the things that used to be deprioritized? Why, with the freed up time, is generating code the best outcome?

These questions likely have different answers depending on organization size but I'm not sure I understand why orgs wouldn't just do more work in this scenario instead of blending responsibilities. It's not like there's infinite mental bandwidth just because an LLM is generating the code

> Code review is being unbundled. Its four functions (mentorship, consistency, correctness, trust) each need a new home.

> If code changes faster than humans can comprehend it, do we need a new model for maintaining institutional knowledge?

The humans we have in these roles today are going to suffer. The problem starts at hiring, because we rewarded memorization of algorithms, and solving inane brain teasers rather than looking for people with skills at systems understanding, code reading (this is a distinct skill) and the ability to do serious review (rather than bike shed Ala tabs over spaces).

LLM's are just moving us from hammers and handsaws to battery powered tools. For decades the above hiring practices were "how fast can you pound in nails" not "are you good at framing a house, or building a chair".

And we're still talking about LLM's in the abstract. Are you having a bad time because you're cutting and pasting from a browser into your VIM instance? Are you having a bad time because you want sub-token performance from the tool? Is your domain a gray box that any new engineer needs to learn (and LLM's are trained, they dont learn).

Your model, your language, your domain, the size of task your asking to be delivered, the complexity of your current code base are as big a part of the conversation. Simply what you code in and how you integrate LLMs really matters to the outcome. -- And without this context were going to waste a lot of time talking past each other.

Lastly, we have 50 years of good will built up with end users that the systems are reliable and repeatable. LLM's are NOT that, even I have moments where it's a stumbling block and I know better. It's one of a number of problems that were going to face in the coming decade. This issue, along side security, is going to erode trust in our domain.

I'm looking forward to moving past the hype, hope and hate, and getting down to the brass tacks of engineering. Because there is a lot of good to be had, if we can just manage an adult conversation!

Here some story. Long time ago, i wrote a (software) accounting system. From 1st principles - nomenclatures, accounts, double-entry, transactions, balance (=current cached status), operations+reports on top of these. 5 tables (+1 for access control later). Very flexible and re-configurable into whatever one imagines. But anyway.

We deployed it at several places. The biggest one - retail with 50+ salepoints across whole region - was the most troublesome.. and after a month+ back-and-forth it dawned that.. they did have very well-working paper system of accounts/documents/data/values flow which was highly optimized for humans and the reality it was in (papers, remote places, delays, etc). Humans forget, make mistakes, displace things etc ; paper rots in time; distances make things out-of-sync - yesterdays invoices from village X will come tomorrow - maybe - .. etc. So their document flow - and even people-roles - were aligned with that system. Duplicating some things and completely avoiding others.

The new software had no such notions. There was no such thing as forgetting, displacing, out-of-balance. And while temporal stuff was fine, the document flow - even if consisting of same dot-matrix-perfect documents - was different to what they have used to. So.. it took them - and us - 3 months to retrain the personnel to unlearn their old system and to start actually using the new one properly, and enjoying the ride instead of fighting it.

Back to the topic.. i guess the old system of software engineering, built last 50+ years, has to be rearranged now. Not everything, but.. quite. Some things probably may wait for tomorrow, as the paper notes, but some - like roles and what they mean, and the cognitive/understanding chasm - is for yesterday..

Edit: after reading the whole paper, i think there are some things that can be "loaned" from hardware-design (chips etc) flows and processes. i see this analogy - the hardware's target environment (actual physical world, e.g. silicon etc) is also non-deterministic.. just mostly. Things like Requirements engineering, design-for-test ; all the enveloping (heat, power etc) and whatever else may come handy (i am not hardware dev, only seen these from aside, e.g. from a Verilog compiler)

Up the abstraction ladder; we conceive axioms and constraints; we define actors and objects; we direct rules, flows, sequences and say when and how each one of them lives and dies.

May you live interesting times (some say this is a curse)

I'll say it again and again and again: Rust is the best language for ML right now.

You get super strict, low-defect code. If it compiles, that's already in a way a strong guarantee.

Rust just needs to grow new annotations and guarantees. "nopanic", "nomalloc", etc., and it would be perfect. Well, that and a macro-stripped mode to compile faster. I'd happily swap AOT serde compilation (as amazing as Serde is) for permanent codegen that gets checked in and compiles fast.

This is a fairly easy to read doc discussing some of the challenges with using AI tooling in a forward thinking and disciplined way. Coming from Thoughtworks it also gives a bit of gravitas and legitimacy.

There's good stuff in here. It would be a shame for the larger HN community to miss out on this conversation.

I don't think the industry is ready or has the manpower to deliver on all the promises they're making and a lot of businesses and people will suffer because of that.

> we have 50 years of good will built up with end users that the systems are reliable and repeatable

There's good yes, but also we've raced to build dystopian bullshit and normalized identity theft because most software is garbage. There might not be as much goodwill as you think. Software eats the world, and many simply feel helpless. The erosion of trust you're predicting has already happened, or never existed IMHO.

LLMs may not 1-shot reliable and repeatable systems, but they're a powerful tool that I hope will end up improving systems overall, for reasons you've mentioned, among others.