Show HN: Insights.hn – Real-time Hacker News posts and comments analytics

Show HN: Insights.hn – Real-time Hacker News posts and comments analytics(insights.hn)

4 points by burmecia 1 year ago | 9 comments

Hi HN, I'm excited to share a project I've been working on for the past few months: https://insights.hn/, a real-time analytics platform for Hacker News posts and comments.

Why did I build this project?

I built this project to help me understand the trends and patterns in the HN community. I've always been fascinated by the discussions and insights shared on HN, and I wanted to create a tool that could help me explore and analyze this data in real-time.

How it works?

The platform collects data from the Hacker News API and processes it in ClickHouse database to provide insights on the top posts, comments, and users on HN. The data is updated in nearly real time, so you can see the latest trends and discussions as they happen.

I hope you find this project useful and interesting. I would love to hear your feedback and suggestions for how I can improve it further.

Previous similar HN threads:

- https://news.ycombinator.com/item?id=3745965

- https://news.ycombinator.com/item?id=34110321

jimmySixDOF 1 year ago |

Love that you have been improving this idea since 2012 !! I think enriching HN is a super interesting problem I have also thought about quite a lot. The realtime angle looks really good fun to see likes/page position as it happens. I notice you show "Top Comments" in the details and wonder how you determine this since afaik there is no concept of likes for individual comments only stories from the API. I need go looking at the API again I found one method somewhere that would return the whole story with all comments fully sorted in order together in a single json so I guess if the "best" comments float to the top of their branch you could track that over time. Just curious and wanted to say this looks amazing !!

burmecia 1 year ago | |

Thanks, the "Top Comments" is sourced from the "kids" field of the "items" endpoint. As the API docs (https://github.com/HackerNews/API?tab=readme-ov-file#items) says, it is "in ranked display order" so I just used it to sort the top comments.

jimmySixDOF 1 year ago | | |

Yah that makes sense and after looking I could not find the API call that pre sorted each thread (it must not have been native algolia or firebase) - and without that the way you are doing it sure beats recursively reconstructing the thread. All the best and I look forward watching any features you add in the future!

zX41ZdbW 1 year ago |

This is really great!

I can suggest more ideas that will be easy to add:

- a spark line or heat map of upvotes for every thread: https://github.com/ClickHouse/ClickHouse/issues/59020

- a built-in SQL editor for custom queries;

If you need help in supporting or hosting it, write to milovidov at clickhouse.com

burmecia 1 year ago | |

Thanks for the ideas, heat map looks great! Will also consider add custom queries, it can be quite useful.

PeterZaitsev 1 year ago |

One more thing, I would suggest to make comparisons linkable - I was looking to share with folks the graph of how MySQL vs PostgreSQL is doing, sadly the URL still links to generic keyword comparison tool

burmecia 1 year ago | |

That's a good idea, will put it on to-do list.

PeterZaitsev 1 year ago |

Very cool. I was surprised though to see keywords seems to be hardcoded, would be great to be able to graph and keywords and phrases. Any decent search engine should be able to support that

burmecia 1 year ago | |

yes, the keywords search is using full-text searching and it is not efficient so I hardcoded them. Will consider switch to use embedding, that will be more efficient and accurate, also can support custom keywords search. Thanks for your advice.