Multi-Query Retrieval

A single query captures just one phrasing — and if the relevant chunks are worded differently, plain vector search can miss them. Multi-query retrieval fixes this by using an LLM to generate several versions of the query, retrieving for each, and merging the results. More angles means higher recall and less sensitivity to how the question happened to be phrased.

💡 In one line: Multi-query retrieval rewrites the query into several variations, retrieves for each, and merges the results for broader coverage.

The Problem It Solves

One query is one perspective. If the user's wording doesn't match the document's wording — a vocabulary mismatch — or the question is ambiguous, a single embedding may miss relevant chunks. Covering the question from several angles finds more of them.

What is Multi-Query Retrieval?

You use an LLM to expand the user's query into N variations — paraphrases, sub-questions, or different perspectives — then retrieve for each and merge + deduplicate the combined results.

Whiteboard
Whiteboard diagram


Why It Works

Different phrasings produce different query embeddings, so they land near different (and overlapping) chunks. Taking the union of their results improves recall and makes retrieval robust to the exact words the user chose.

Related Techniques

  • Query expansion / rewriting — the general idea of reformulating queries.
  • RAG-Fusion — multi-query combined with Reciprocal Rank Fusion (RRF).
  • Sub-question decomposition — break a complex question into parts, retrieve each.
  • HyDE — generate a hypothetical answer, embed that, and retrieve with it.

Trade-offs

  • An extra LLM call to generate the variations (latency + cost).
  • More retrieval calls, and you must merge/dedupe.
  • Variations can drift off-topic and add noise if not constrained.

Best Practices

  • Generate 3–5 focused variations.
  • Dedupe or use RRF to combine.
  • Rerank the merged set and cap total context.
  • Use it for ambiguous or multi-part questions.

Implementation

Available in frameworks — for example LangChain's MultiQueryRetriever, and the RAG-Fusion pattern for RRF-based merging.

Summary

  • Multi-query retrieval rewrites one query into several, retrieves each, and merges.
  • It fixes vocabulary mismatch and ambiguity, boosting recall.
  • The union of results covers more relevant chunks than any single phrasing.
  • Related ideas: RAG-Fusion (RRF), sub-question decomposition, and HyDE.
  • Costs an extra LLM call — so generate a few variations and rerank the merge. EOF echo created