Our first outage from LLM-written code(sketch.dev) |
Our first outage from LLM-written code(sketch.dev) |
> This kind of error has bitten me before, far before LLMs were around. But this problem is exacerbated by LLM coding agents. A human doing this refactor would select the original text, cut it, move to the new file, and paste it. Any changes after that would be intentional.
huh! it's almost like a human SHOULD make that change...
> // Log error but continue with other installations
was this comment also written by LLM? if so, maybe simply stripping comments from the codebase the LLM works on would help? I've not studied enough if comments are more helpful or toxic for context. I do know that they rot...
I'd love for "the CEO logged in" to be somehow part of the root cause...
In the code fragment shown on the blog post, the problem would only manifest itself when the "ListUserRepos" call on "ghClient" failed. The CEO account probably lacked some configuration necessary for that call to work (my guess would be that it did not have a valid GH account associated).