Show HN: Fixing AI memory blind spot on connected facts with benchmark

Show HN: Fixing AI memory blind spot on connected facts with benchmark(yourmemoryai.xyz)

7 points by SachitRafa 8 days ago | 2 comments

Semantic search alone is not enough to capture all connected facts, they will capture the semantically most identical memory only.

Tested on HotpotQA public dataset:

Vector + BM25 + entity graph: BothFound@5 71.5% Vector + BM25 only: BothFound@5 59.5%

Entity graph is the game changer to extract connected facts.

More Benchmark result:

LongMemEval-S: 84.8% recallAll@5 LoCoMo-10: 59% vs zep cloud 28%

What is your approach for connected facts retrieval ?

noashavit 7 days ago |

This looks helpful. Context is the main issue agents face. How does this fix issues with messy data though? Like in the real world when you have 2 reports for the same metric with different results. Will the agent have enough context to provide the same, accurate, result consistently?

SachitRafa 6 days ago | |

So based on the importance that it gives the reports individual will decide that which report will survive in the system. You can manually also identify and set the importance or can let llm be the deciding factor here to assign weightage.