Show HN: Pip_search revived: CAPTCHA-less challenge solver, significant speedup As `pip search` has been dead for years, `pip_search` by @victorgarric filled that gap for me and despite doing PyPI web parsing, worked quite well. The cat-and-mouse game between PyPI and the tool developer already lead to breakage of the tool in the past, and recently it happened again: PyPI moved to a newer anti-bot flow using PAT (Private Access Tokens) by Fastly, beyond the earlier, simpler PoW checks. I decided to give it a try and bring the tool back to life. And it worked out! For reference: Fastly CAPTCHA-less login was presented at Apple WWDC in 2022 with follow-up article by Fastly themselves: https://www.fastly.com/blog/private-access-tokens-stepping-i... What I did: - Deminified the challenge script and traced execution with headless Chromium (`zendriver`). - Found that Chromium was being routed to a downgraded, solvable PoW with two postbacks. - Reproduced that downgraded flow in pure Python by matching the relevant fingerprint behavior. No Chromium dependency. (This would've been nasty!) Codebase has increased in size, so, in order to make it manageable and maintain the quality, I significantly refactored it, introducing properly designed solver pipeline (PAT/PoW/metrics), stronger tests and modernizing project layout/tooling. I also switched networking to `httpx` which, after some tuning, showed a fair speedup. Feel free to give it a try: uv tool install 'pip_search @ git+https://github.com/ink-splatters/pip_search' I also submitted a PR and hope for the changes to be eventually upstreamed. My final thoughts on this: Honestly, it feels both rewarding and... weird that downgrading the challenge is a viable workaround tactics. I would love feedback from people familiar with Fastly challenge internals! |