This is something that has bothered me for quite a while, and I don't see a lot of people talk about it: Agents, in most cases, impersonate the human operator, by design, with no way to enforce, disclose, or control it. I believe this is causing an illusion of human in the loop, and is not intentional, and should be discussed.

For example:

All commits, pushes, PRs, and PR comments are all going to appear as the developer whether they wrote them or not. (You may have Co-authored-by, but not everyone has it set up).

The good: you are accountable for what your AI wrote.

The bad: while everyone should assume you used AI these days, there is still an expectation of some human-in-the-loop.

When your agent uses the GitHub MCP or CLI, it's most likely using an OAuth authorization (even if it's a GitHub app, you also give consent for it to act on your behalf)

This allows the agent to open PRs as you (which is intentional to force a 2nd reviewer. While you should review "your" own code, especially if AI wrote it, you shouldn't be able to also approve it). But it also allows it to comment or event approve PRs as the developer.

This indirectly means that we allow AI to review and approve its own code with an illusion of a human in the loop without leaving any traces.

E.g. Alice creates a PR using Claude Code (either locally or via the web).

Bob "reviews" it by checking out the branch and prompting their agent to run the /review-pr skill it helps with his token quota and leaderships expects 10x more features so he doesn't have time to actually read the code...

Since he has the GitHub MCP / CLI, this looks as if Bob wrote the comments (let's say they have a system prompt that removes emojis and em dashes... it will pass a turing test, that's if a human would have been reading his PR comments in the first place, but I'm getting ahead of myself). There is no explicit control that says they must disclose this is not really them who did the review, (and if there is, how would you detect or enforce it?)

Alice receives Bob's feedback + feedback from various other "AI Code Review" tools. She also needs to be tokenmaxing, so she asks her agent to /answer-pr-comments (fixing, or replying to comments as her)

Bob receives that, asks their agent to review Alice's responses and resolve the comments if they are addressed, or add more comments if anything was missed. (/re-review-pr skill)

You can use your imagination to see where this is going...

So at the end you can have a feature released to production where

- AI wrote the code

- The same AI (as in same model+harness) reviewed its own code (via a "PR")

- AI reviewed the review of the code and fixed / pushed back

- AI reviewed the review of the review, saw nothing was left and approved the PR (Bob asked Claude to "If you think it's prod-ready, approve it", nothing in the approval shows that Bob didn't even read the mermaid diagram or TL;DR summary of the PR...)

- CI passed the tests that AI wrote and AI reviewed

- AI auto-generated the documentation

- QA did "manual browser testing" by using computer use and a markdown file of test cases that AI generated, and confirmed manual testing is done

- E2E tests that AI wrote also pass so there is "no regression"

- code was shipped to production

- code initially works, but becomes slowly unmaintainable due to context rots, duplication, and eventually breaks in production

All audit trails show humans involved in various checkpoints of the feature. But all of this can happen without any of them doing anything but accept all changes (Simpsons depicted it great here: https://www.youtube.com/watch?v=R_rF4kcqLkI).

No one really asks developers explicitly not to do it, on the contrary, they are being asked to use AI more to produce more, so they do.

Is it just me who is worried about it?

user:	goyozi
created:	March 2, 2022
karma:	15