Ask HN: What is going on at archive.ph? The queue has been in 5 digit numbers for a few days. And for a few weeks before that it was possible to get articles archived by archiving them when americans are sleeping (at GMT morning). Nowadays it is no longer possible and the waiting line is about 2-3 days. What on Earth is going on? |
https://github.com/webrecorder/archiveweb.page
I also convert web snippets I find useful to the markdown format and store it in my Joplin notebook, so that way it lives on even if the website is gone.
Can you run your own instance of archive.ph? Obviously not their software (they don't release it), but is there a version you can run yourself?
The important parts of what archive.ph provides are:
1. Headless chromium that works around 99.9% of the "headless fingerprinter" techniques. Yes, they do actually spin up a full browser engine; that's what sets them apart.
2. Decent curated collection of paywall-bypass tactics.
#2 is definitely available from other sources. So really what we're looking for is a one-click "headless chromium that deals with 99.9% of the fingerprint detectors".
I'm not terribly motivated right now to dive into the community and ask for help, so I won't seriously complain, but it's at least not trivial to use.
Web archival is a very, very complex and deep topic. If you're interested in it, the archiveteam's wiki is a good resource. Start here, I think:
https://wiki.archiveteam.org/index.php/Introduction