Advanced RAG retrieval: parent-child, hybrid fusion, and two-stage reranking

The RAG-fundamentals and RAG-improvement posts get you a working system; document-heavy domains (legal, technical, scientific) need more. When retrieval quality is the whole ballgame and the documents are long and structured, a few retrieval-architecture moves separate a demo from something you'd stake a decision on.

Chunk small, return big

There's a tension in chunking: small chunks match a query precisely but lack surrounding context; large chunks carry context but match fuzzily. The fix is to decouple what you match from what you return.

• Parent-child (small-to-big) — embed and match on small chunks, but return the parent section they belong to, so the model gets precision in retrieval and context in the prompt.
• Sentence-window — embed individual sentences, then return a window of the surrounding sentences around the best match.
• Section-aware chunking — for structured documents, split on the document's real structure (abstract, spec, claims, drawings) rather than fixed character counts, so a chunk is a coherent unit.

Hybrid search: BM25 + dense, fused

Dense embeddings capture meaning but miss exact tokens — part numbers, statute codes, precise terms — while BM25 nails lexical matches and misses paraphrase. Run both and fuse the results. Reciprocal Rank Fusion (RRF) combines two ranked lists by summing 1/(k + rank) for each document (with k around 60 as a damping constant), which needs no score normalisation between the two very different scorers — a real advantage, since BM25 scores are unbounded and cosine scores sit in [-1, 1], so their raw values aren't comparable.

Weighted fusion vs RRF, and tuning the mix

RRF fuses by rank, which sidesteps the incomparable-scales problem entirely. The alternative is a weighted linear combination — score = α·dense + (1−α)·sparse — worth reaching for when your query distribution is stable and known, when you want to preserve the raw confidence magnitude rather than flatten everything to ranks, or when your vector database supports weighted fusion natively. The catch: you must normalise the two score scales first (min-max or a sigmoid), or the unbounded scorer dominates. And you can tune α by query type — weight sparse higher for entity-heavy queries (a flight number, an airport code, a product SKU) and dense higher for intent-heavy, experiential ones ('a quiet beach resort for a honeymoon'), routing each query with a lightweight intent classifier at the gateway.

Two-stage: retrieve wide, rerank narrow

An ANN/bi-encoder search is fast but approximate — it embeds query and documents separately. A cross-encoder reranker reads the query and each candidate together and scores relevance far more accurately, but it's expensive: one model pass per candidate (the bi-encoders-vs-cross-encoders post explains exactly why). So you use both. Retrieve a wide shortlist cheaply (say top 100), then rerank it down to the best few (top 5). The economics work because reranking cost is bounded by the shortlist size — keep it capped.

You have options for the reranker, not just one vendor. A hosted reranker like Cohere Rerank is the easy path; an open-source cross-encoder like BGE-reranker runs on your own hardware when data can't leave your boundary or you want to cap cost; and for small shortlists an LLM-as-judge can filter for relevance directly. Match the choice to your privacy and cost constraints, not just the leaderboard.

Contextual retrieval: don't strand a chunk from its document

A subtler failure than boundaries: a chunk that's clear in its document but ambiguous alone. 'The revenue grew 12%' means nothing retrieved without knowing which company and which quarter. Anthropic's contextual retrieval fixes this by prepending a short, generated blurb of context to each chunk before embedding it — 'This is from Acme's Q3 2024 filing; the section discusses…' — so the chunk carries its own context into the index. It measurably lifts retrieval, and it composes with hybrid search and reranking rather than replacing them.

High recall vs high precision

The retrieval goal isn't universal — it depends on the task. A chat assistant wants high precision: the top few results must be right, because that's all the model sees. A prior-art or discovery search wants high recall: missing the one relevant document is the failure, and some noise is acceptable. Tune k, fusion weights, and reranking for the goal you actually have, not the default one.

The failure modes to design against

• Lost-in-the-middle — a relevant chunk buried in the middle of a long context gets ignored; put the best chunks first (which is exactly what reranking buys you).
• Chunk-boundary loss — the answer is split across two chunks and neither is retrieved whole; overlap and parent-child retrieval mitigate it.
• Stale index — the source changed but the embeddings didn't; re-index on change, and treat the index freshness as a monitored property.

Basic RAG asks 'what's similar?'. Production RAG asks 'what's similar, phrased how the corpus phrases it, ranked by a model that actually read the query' — and only then hands the model an answer.