Broke Prod

2 points by cloaky233 8 days ago | 2 comments

We had 3 different stages of code reviews and something of a stripe minions thing running as well, but one small code change broke prod.

What's the use of code reviews if they just keep putting p2 p3 comments and quietly let p0 issues be shipped..... :C

manos-saratsis 8 days ago |

You need to have multiple specialised agents, all with the ask to check probability and impact, the latest versions of frontier models do a good job to find out. The most important is harness engineering, what we do at dromeas.ai to verify the code that goes to prod is a)have the code mapped before hand for the right agentic context, b)specialised agents c)an LLM council as 2 frontier models are hard to miss d)deduplication step . All before assessing a PR, a commit, a release. It is not easy, harness engineering is tough

laysheth 8 days ago |

Code reviews are static, without having agent's reasoning, it's not quite possible to know why a change was done. without that ig there's no way you can potentially find issues that do not appear in code, but appear during runtime :)