Imagine a corpus of documents with scientist biographies. The traditional RAG works fine until you ask questions like: - "Who was born before 1800?" - "How many are mathematicians?" - "List names and birthdays for mathematicians" These result in an incomplete answer due to top-k, with no signs of incompleteness. For an initial corpus, it is possible to improve this problem by extracting metadata for a predetermined set of fields. This approach has two problems: - One has to predict all the questions that can be asked against the corpus upfront. - Constantly revising that prediction as the documents change, e.g. adding Nobel prizes later, or extending the document set to contain artists. DuoRAG aims to solve both problems by: - An initial metadata (schema) discovery before the first ingestion - Self-update schema with candidate fields when it fails to answer a question |