Recall (RAG Evaluation)

In RAG, if the retriever doesn't surface a relevant chunk, the LLM can't use it — so retrieval recall often sets the ceiling on answer quality. Recall measures completeness: of all the relevant chunks that exist, how many did you actually retrieve? It's usually the first thing to check when a RAG system gives incomplete answers.

💡 In one line: Recall measures how many of the truly relevant chunks the retriever actually found.

What is Recall?

Recall is the fraction of all relevant items that were retrieved — it answers "did we find everything relevant?"

Recall=relevant retrievedall relevant\text{Recall} = \frac{\text{relevant retrieved}}{\text{all relevant}}

The Formula

In confusion-matrix terms:

Recall=TPTP+FN\text{Recall} = \frac{TP}{TP + FN}
  • TP (true positives) — relevant chunks that were retrieved.
  • FN (false negatives) — relevant chunks that were missed.

Recall @ k

In practice you measure recall over the top-k retrieved results — context recall @ k. A larger k generally raises recall (you cast a wider net), but there are diminishing returns and a cost.

Why It Matters in RAG

A missing relevant chunk means the LLM literally cannot answer from it — no amount of prompting fixes absent context. Recall is often the biggest lever on RAG quality, so fix retrieval before blaming the LLM.

Recall vs. Precision (Preview)

  • Recall — of the relevant, how many did we retrieve? (completeness)
  • Precision — of the retrieved, how many are relevant? (cleanliness)

They trade off: raising k improves recall but usually lowers precision. Precision is the next subtopic.

How to Measure

You need ground-truth relevant chunks per query (labelled), or an LLM-as-judge to decide relevance. Frameworks like RAGAS provide a context recall metric to automate this.

Improving Recall

Whiteboard
Whiteboard diagram

Ways to raise recall: a better embedding model, hybrid search (catch exact terms), multi-query retrieval (more angles), better chunking, or a larger k.

Example

Suppose 5 relevant chunks exist for a query and the retriever returns 3 of them:

Recall=35=0.60\text{Recall} = \frac{3}{5} = 0.60

Trade-offs

Chasing high recall with a huge k pulls in noise, which hurts precision and can confuse the LLM. Balance it with reranking to keep the top results clean.

Summary

  • Recall = relevant retrieved / all relevant — a measure of completeness.
  • In RAG it often sets the ceiling on quality: missed chunks can't be used.
  • Measure recall @ k against ground-truth relevant chunks (e.g. with RAGAS).
  • Improve it with hybrid search, multi-query, better chunking, or a larger k.
  • It trades off with precision — the next metric. EOF echo created