Show HN: Sourceminder.org - token-efficient code search(sourceminder.org) Hey HN, The code indexing tools released in December now have added capabilities (Rust, Perl support), better token efficiency, easier install method, and a website! The website has a wasm port of the query tool, qi, so you can try it out in the browser. Let me know what you think. Thanks! |
The 15% number is based on two runs from the ./experiment folder, on an instance from the SWE Bench Pro benchmark. I picked two trajectory logs, from the pro_pilot_openlibrary_haiku_v2 folder, that I consider to be representative of the token savings one can expect. There's huge variance just between individual runs of the same model on the same instance, though, so it's somewhat difficult (and expensive, at API rates) to cut through the noise. But I ran over 80 trials, and I believe that number is accurate.
And I'm publishing the "token science" research in the repo itself, so anyone who's curious about it can review the logs and inspect the python scripts that I used. And if someone out there has the time, inclination, and money, to run some of those experiments themselves, it'd be great to get consensus on some of those results.
And if you'd like to share feedback directly, rather than publicly, my email is on my profile page.