Shutting down our search proxy Leta(mullvad.net) |
Shutting down our search proxy Leta(mullvad.net) |
The main problem is that Google desperately wants you to run their JavaScript. They implemented a JS requirement a few months ago, but you could still pretend to be the Lynx web browser and then not be served any JS. Then a little bit ago they removed Lynx support, so that stopped working. There were a few other things you could pretend to be that still worked though, like an old iPod I believe. Then those were blocked as well, but currently there is still at least one user agent string that works.
Some of this I learned about from a proxy called 4get.ca. The guy who runs it is clever and has the energy (for now) to keep the Google scraper working. The caveat though is that he has a somewhat crude and demented sense of humor.
Even if all the user agents are blocked, I worked with him a bit on an alternative bypass which is a bit slow, and is based on my article in the most recent issue of 2600 Magazine (Autumn 2025 if you want to read it). If need be I can probably make it faster. He also has a bunch of Google CSE api keys and makes that available as a user selectable option.
It isn’t a meta-search engine because it doesn’t mix results together from different sources and re-rank them, it’s just a proxy, but as a proxy it’s pretty good.