Meta has tried to scrape this site 1 million times in 2 weeks(robertmay.photography) |
Meta has tried to scrape this site 1 million times in 2 weeks(robertmay.photography) |
Meta presumably has some at least marginally competent people on hand that don’t need random bloggers to tell them to sort out their scraping tech…
It's up to 1.5 million requests now, sigh. My guess is that Meta has too much money and is in the "move fast and break things" stage where they're just throwing money at a problem. Not setting a max crawl depth is hilarious though.