My apologies, please allow me to clarify. This is not for posts submitted. I would want the ability to send the HN comment contents and the prompt to any public LLM model; enough requests and models and you’ll converge somewhere near Truth. One model biased? Certainly possible. All models biased? Unlikely.

This would be a public HN comment evaluation and verification engine in the browser. One could go so far as to build a website based on the concept, consuming the HN comment firehose and scoring comments as they’re created, an append only transaction log of sorts. Comments are permanent after two hours, so you only have to run each comment through each model once if you wait until that expiration window passes.

“Computah, validate truthiness of all comments in this thread.”

(Bonus points for scoring HN users by how factually accurate they are over time from their comment history)

How would you design the pipeline and user interface to reflect the temporal nature and potential theta decay of evaluations? Reprocess comments on a cadence?

For example, however you represent a comment, it could then support expansion of a tree to each evaluation snapshot point in time, like expanding a directory listing its files. My thought on this is somewhat similar to Wayback Machine’s interface per URL; the URL remains static over time, with a calendar and indicators for date of snapshot, and drill down into each snapshot.

In this use case, the comment is the URL, and each LLM evaluation of the comment is a snapshot. Append only.