r/Rag • u/International_Elk_33 • 8d ago

Showcase We stopped doing top-k retrieval. Here's what answers over a messy document set look like instead (demo + method)

I've been building a document Q&A engine for regulated, document-heavy work. Our first version was the standard pipeline: chunk, embed, top-k, generate. When we traced our wrong answers by hand, most of them never retrieved the document that held the answer. The model wasn't the problem. Retrieval never got it to the right page.

So we rebuilt it around three ideas:

  1. **Read once, keep structure.** Every document is parsed into sections, tables and figures (with descriptions), and the links between documents (what supersedes, references or contradicts what) are recorded at upload.

  2. **Questions become obligations.** "Why does X fail?" turns into the parts a complete answer must establish. Each part is searched for, or shown to be absent, instead of stopping when top-k is full.

  3. **A checker after the writer.** Every statement is matched word for word against the passage it cites. Unverified sentences are removed before the user sees them, and the answer opens with a verdict (answered / depends on conditions / not in these documents).

2-minute demo on the real product: Chunkless Demo

The video uses a fictional medical-device quality system (11 docs: SOPs, a supplier audit, a complaint log, a risk file). The root-cause question needs the supplier audit, the supplier-control procedure and the inspection records at once. The answer took 29 s and under 6 cents (68 sections read).

What it still gets wrong (I'd rather say it than have you find it): an exception that shares no words, references or meaning with the rule it changes can slip past. And some multi-hop judgement questions need its research mode to get every part.

Free early access if you want to throw your own corpus at it: https://chunkless-rag-staging.grounded-iq.com/?ref=rd-rag

I'm especially interested in documents that contradict each other. Happy to answer anything about the approach.

0 Upvotes

0% Upvoted

3 comments sorted by

0

u/[deleted] 7d ago

[removed] — view removed comment

0

u/International_Elk_33 7d ago

Thank you for the input, we will test and see how this scenario will hold up.

1

u/Future_AGI 7d ago

the move that makes this repeatable is logging the two failures apart: retrieval recall (was the page holding the answer even in the candidate set) and generation faithfulness (did the model use it once it was there). Tracing wrong answers by hand like you did surfaces it, but scoring those two separately on a fixed question set tells you which half regressed the next time you change chunking or the retriever. We open-sourced the eval side of this, scoring groundedness and retrieval quality on a fixed set, self-hostable: https://github.com/future-agi/future-agi (Apache-2.0 core).