Hacker_news_scraper

1 points by vstuart 6 years ago | 2 comments

hacker_news_scraper | https://github.com/victoriastuart/hacker_news_scraper

A Python 3 script for scraping the Hacker News feed, filtering that content by

* number of points, and/or

* number of comments, and/or

* excluding posts {dead | flagged | youtube | wikipedia | ...} according to a keywords list

Run via ~/.bashrc alias or crontab.

Experimental code; I will monitor it locally (scheduled to run {06:00 | 18:00} daily, starting 2020-04-14) for bugs, etc.

Etheryte 6 years ago |

Not to discount your effort, but is there a benefit for scraping the data when HN offers an official API[1]? Does the API not expose all the data you need?

[1] https://github.com/HackerNews/API

vstuart 6 years ago | |

Good to know; thank you. :-) I suppose the advantages (mine) may include keyword exclusions (I'm not sure if that functionality is available in the API), and local hosted (hence, more "private" -- whatever that means these days ;-) code.