I imagine this is similar to how Twitter's/X's community notes work. Something along the lines of when you have two accounts that disagree traditionally and they agree on something. That's how you know it's likely to be true.
So from that approach, you actually want to have two AIs that are on the other side of the spectrum of whatever you're trying to find out if it's true e.g. if a conservative AI and a liberal AI both agree that something is false, it's highly likely to be false.
I don't think LLMs can discern truth when the majority of news sources are incentivized primarily for views.
Thanks for that, btw! At least someone questioned it :)
What would really set it apart is adding fact-checking before publishing and letting it post directly to TikTok. That combination would make it something I'd actually use every day.
Checking before you publish - that's interesting usecase I haven't thought about. Nothing stops you to fact-check your own content and decide what to do with it. Point is a report itself not more video content from it
In the age of ai, where you can flood the zone with easily generated content, even a democratic society might struggle to maintain an stable sense of shared reality and history
But there is a room for improvement, what do you suggest? Have a Judge agent which will check results?
But, there is a reason why it's not an extension. At least for now. Whole product isn't a UI or service, it's a set of skills for your AI agent of choice, which gets content from URL or whatever shared as a source, splits it into claims and does extensive web searches for every claim to compare it with statement from provided content.
On local models constraint isn't a model itself - but search. Model don't judge from it's trained memory, so even local model will need a backend for search, otherwise it can't provide a verdict.
Apparently the "AI-generated bullshit" detection is already part of LLMs.
HN’s community has built-in bullshit detectors.
Hmm, smells AI generated. Why should an LLM (this is from a SKILL.md) care about $LINES?
but with "AI" now it's possible not only to do non-stop but in REALTIME
you could even just restrict the source of the check to the paper's own reporting the past fifty years
* https://www.washingtonpost.com/graphics/politics/trump-claim...
I'd like to see that backfilled, all the way back to the "long form birth certificate" (remember that horror show)
PS I've always assumed all politicians lie, I'm sure you don't think Pelosi or Bush Jr for example were some paragons of truth. Doesn't make it OK, but have some perspective.
the entire premise of a "bullshit detector" that's entirely vibe-coded is laughable
I don't mind that I've used AI to build this, and I'm open about it ;)
- production grade app generator without AI :)
What's dangerous is treating information from an anonymous author as factual. You gotta check that in other ways.
It can be, but that's hard to correctly judge on at scale.
> I've always assumed all politicians lie
Everyone lies at various points to some degree or another, ranging from well-intentioned to malignant, and it takes tremendous sleight of hand to pretend that they're even close to comparable.
> have some perspective.
Indeed. Perspective: https://en.wikipedia.org/wiki/False_or_misleading_statements...
> Doesn't make it OK, but
Why the "but"? Are you justify something? The difference between your framework and mine is that my framework statement ends with a period after "Doesn't make it OK."
I'll work on empirical sources improvement in a next releases.
> Something along the lines of when you have two accounts that disagree traditionally and they agree on something. That's how you know it's likely to be true.
Outfits that blindly parrot talking points would not be at odds and thus not be good candidates.
Sounds challenging to me...
Will read reliable-sources page properly.