First-stage retrieval should be fast and broad; a reranker can spend more computation on a small authorized candidate set. Query rewriting or decomposition can clarify intent and improve recall, but generated rewrites may drift or inject assumptions. Preserve the original query, constrain transformations, fuse evidence, and evaluate every added stage against quality, latency, cost, and safety.
Stage one retrieves perhaps 50–200 candidates with lexical, vector, or hybrid search. Stage two scores query–candidate pairs with a cross-encoder, small model, or domain rule and returns the final 5–15. Reranking cannot recover a relevant document absent from the candidate pool, so first-stage recall remains essential.
Keep original and transformed queries, cap their number, and never let a rewrite broaden the caller's authorization.
Assign a strict latency budget. If rewriting or reranking times out, return the reviewed first-stage result or a safe unavailable state. Circuit-break degraded external models. Record which path served the response. Do not retry a slow reranker until the user-facing deadline is already lost.
Run ablations: baseline, rewrite only, rerank only, and combined. Measure retrieval metrics, final answer metrics, p95 latency, calls, cost, and failure rate. Manually review query drift, negation, names, numbers, and security-sensitive queries. Use a held-out test split.
Design an ablation experiment for query rewriting and reranking on this retrieval system [details]. Define candidate depth, transformations, fusion, model versions, timeouts, fallbacks, privacy controls, drift review, quality metrics, and latency/cost budgets.
Verification contract: Keep a transformation only if it shows a reproducible slice-level gain without violating security, latency, or cost gates.