The data was collected for a paper that was attempting to find a lexical difference between human contributions and LLM contributions. The reason for this was because repository maintainers need to make sure submitted code has been at least reviewed by a human. By detecting unattributed AI submissions, it can tell maintainers that they need to pay extra attention to the repositories.
There has been a persistent attack on GitHub repos by agents trying to land a commit to become a contributor. The default GitHub permission is to allow contributors to execute actions against the repo’s compute budget, so it could be a botnet. Another possibility is that they are trying to get a Copilot grant. Someone claiming to be in academia was surveying repos when that program first started. I’m guessing the permissions for that are equally as naive.
This surprised me as well when I wrote the paper! I started data collection for the AI assisted repositories on May 1, 2025 which is the release data of claude code. The reasoning behind this is noted in the paper but can be broken down to, it would be easier to find a large quantity of AI repositories after vibe coding took off. The majority of these repos being AI adjacent topics was an interesting find, but supports the idea that a large chunk of our LLM advancements are being helped by LLMs.
You can't tell if a human has reviewed AI-assisted / generated code. But you can tell if an AI wrote a comment or commit message. The evaluation section of the paper goes over in-depth the findings: https://github.com/ActuallyTaylor/strata/blob/main/paper/Tay...
When I suspect an agent opening a PR I check their GitHub activity. They have usually opened dozens of PRs. Most of them are ignored, high profile packages close them, and AI related projects blindly merge them. They also tend to create dozens of private repos and commit to them non-stop.