Ask HN: When AI writes and reviews code, what do humans do?

1 points by pomarie 350 days ago | 2 comments

I’m noticing more and more teams are using cubic (an code reviewer – I’m the founder) to review PRs written by AIs (eg. Devin, Codex…).

Some teams already run an entirely AI-driven pipeline where AI writes the code, reviews it, and humans just click Merge at the end.

Interestingly, the AI reviewer often finds errors in AI-generated code. For now, AIs can review AIs code with OK results – probably better than a junior-mid level dev.

Qs:

1. Traditionally, code reviews are how junior developers learn. Have you noticed changes in how they learn since codegen has become a thing?

2. Society expects self-driving cars to be dramatically safer (e.g. 10x better) before accepting them. Do you expect similar standards from AI code reviewers, or is slightly better than human good enough?

3. As AI surpass humans at writing and reviewing code (at least for technical correctness), how do you see the role of code reviews changing?

4. Philosophically, do you think code generation AIs will ever become so effective that specialized AI code reviewers won’t find any issues?

ben_w 350 days ago |

For #1, code review hasn't really existed long enough to be a "tradition" — everyone does this differently.

For example, my internship was "here's a book on python, get some stuff done"; and my first few jobs out of university included one where I had exactly one mentor review my code for a few months before they left, a second where I was the only person who knew how the platform worked, and a third where my theoretical peer was someone who refused to listen or share knowledge. And all that was about 15 years ago.

For #2, society has weird risk responses. It's not that we "expect" self-driving cars to be 10x better, it's that we need them to be 10x better just to feel as if the risk is the same when we're not in the driving seat.

Code generation doesn't need this.

Bluntly, most managers don't have much reason to care about code quality already — I suspect many categorise us devs talking about "technical debt" (and similar) in the same way they categorise free yoga classes and table tennis sets in the office: a cost of hiring devs, but not a real business requirement.

For #3, when they're better in general, all bets are off. It's like chess engines: we go from being in control, to helping spot mistakes, to just getting in the way because we "hallucinated" something that wasn't really a consequence of the code or of the ticket or etc.

For #4, we will always find fault, but it will become asymptomatically rare for the faults we find to be both real and not merely aesthetic.

PaulHoule 350 days ago |

Personally I would tolerate AI code reviews if they picked up, say, 50% of quality issues and didn’t waste my time much with BS.