Ask HN: Web scraping based on Computer Vision? Who's developing an approach to web scraping based on computer vision (CV)? I've looked for this but so far not found much beyond [0] -- although the motivations for this are also touched upon by [1]. Scraping is an arms race, of course. A simple but often successful way to fight scraping is, for instance, changing the names of classes routinely. This affects scrapers more than users, because a user doesn't see those class names, while a scraper relies on them. Is anyone scraping using only (or mostly) computer vision on the rendered browser screen, and simulating mouse clicks and key presses? It seems like anti-scraping measures to defeat an CV-based approach would be more intrusive to the user and thus they would be used less often. [0] https://github.com/jimbobewenhall/OpenCV-website-scraper [1] https://incolumitas.com/2021/05/20/avoid-puppeteer-and-playwright-for-scraping/ |