RAG Retrieval, Reranking, Citations, and Grounding
A RAG query pipeline authorizes the user, interprets the question, applies tenant and access filters, retrieves lexical and semantic candidates, reranks them, packs a bounded context, generates an answer, and verifies citations and groundedness. Retrieval evidence improves answers; it does not make model output automatically factual.
What will you be able to do?
- combine lexical and vector retrieval;
- enforce metadata filters before content disclosure;
- rerank and diversify evidence within a token budget;
- return citations that map to exact source passages.
What is the retrieval sequence?
- Normalize the query and detect unsupported or sensitive intent.
- Construct verified tenant, role, document, language, and freshness filters.
- Optionally rewrite into bounded search queries; preserve the original.
- Retrieve lexical and vector candidates independently.
- Fuse rankings, deduplicate near-identical chunks, and diversify sources.
- Rerank a bounded candidate set.
- Pack the best evidence within a context budget.
- Generate with source IDs and an instruction to abstain when evidence is insufficient.
- Validate that every citation exists and supports the associated claim.
- Record retrieval and answer evaluation evidence.
Hybrid retrieval helps exact identifiers and terminology that embeddings may miss. Reciprocal rank fusion is a practical starting method because it combines ranks without assuming scores share a scale.
How should citations work?
Assign opaque source tokens such as [S1] to selected chunks. The model can cite only supplied tokens. After generation, parse citations, reject unknown tokens, link each token to document/version/location, and optionally run a support check. The UI should show source title, exact passage, freshness, and access-safe link.
No citation is better than a fabricated one. If evidence is weak or contradictory, return a bounded uncertainty response or route to a human.
How do you evaluate retrieval separately?
Measure candidate recall, mean reciprocal rank or normalized discounted cumulative gain, filter correctness, latency, and cost. Then measure answer groundedness, citation precision/recall, completeness, usefulness, and abstention quality. Separating stages reveals whether to fix ingestion, retrieval, or generation.
Why does this matter in AI-native systems?
RAG is a data-access feature. Query rewriting and reranking are themselves model operations that need budgets and evaluation. Retrieved text remains untrusted and cannot override system policy or tool permissions.
Common mistakes
- Retrieving globally, then filtering the top results by tenant; authorized evidence may be excluded and signals may leak.
- Sending too many chunks, increasing cost and confusing the model.
- Using similarity score as a universal confidence value.
- Citations point to a document but not the supporting passage/version.
- Evaluation contains only answerable questions and never tests abstention.
- Conversation history silently changes retrieval query across tenants.
AI Pair-Programmer Prompt
Exercise
Define retrieval for “Why did my reset link expire?” using lexical and vector candidates. Include one stale source, one inaccessible tenant source, and one contradictory source.
Acceptance criteria: inaccessible content never becomes a candidate, freshness policy is visible, citations resolve to exact passages, contradiction triggers a qualified answer, and insufficient evidence abstains.
Knowledge check
- Why combine lexical and vector retrieval?
- Should filtering happen after global retrieval?
- Does a citation token prove support?
Answers
- They capture complementary exact-term and semantic matches.
- No; access filters must constrain retrieval itself.
- No; it must resolve to supplied evidence that actually supports the claim.
Completion checklist
Primary references