The most accurate and cheapest AI for scraping(ortutay.substack.com) |
The most accurate and cheapest AI for scraping(ortutay.substack.com) |
You need to write system prompts for Gemini and I bet it will blow smoke chatgpt mini in benchmarks.
Unfortunately the bigger issue with Gemini is cost, which is too high for the scraping use case.
Basically, the Foxtrot scraping library sends the page in chunks. The chunk size is capped at the max context length of each model, which for Gemini is 1,000,000 input tokens for lite. That's compared to 128,000 for GPT-4o-mini.
Typically, you won't need all the tokens in the page, and sending a million tokens when 100,000 will work is wasteful in terms of cost and runtime, and can also hurt accuracy.
I'm going to re-run the benchmarks with a cap on the prompt size for models like Gemini.