We ran 10 models over the same 10 pull requests, each carrying one planted access-control bug (IDOR, missing auth, broken authorization), five times per model, and scored every finding against the actual code.
The result is a clear cost/quality frontier. Open AI’s GPT-5.6 Sol dominates. It found all vulnerabilities at a very affordable $0.70 per PR. For comparison Fable only found 88% and cost $3.61 per PR. No Anthropic model reaches the frontier at all.
This is NOT a harness quirk, we tested and found similar results in the Claude Code and Pydantic harnesses.
Details of our methodology and synthetic repos in the publication.