Query Engines (in LlamaIndex)

A query engine is LlamaIndex's end-to-end RAG interface: you give it a question, it retrieves the relevant nodes, synthesises an answer, and hands back a response with sources. Where a retriever only fetches, a query engine answers. It's the abstraction that makes index.as_query_engine().query("...") a complete RAG system.

💡 In one line: A query engine takes a question, retrieves relevant nodes, and synthesises an answer with sources — retrieval plus generation in one interface.

What is a Query Engine?

A query engine is an end-to-end interface over your index: question in → answer out. Internally it's a retriever (fetch nodes), an optional node postprocessor (filter/rerank), and a response synthesiser (compose the answer).

Retriever vs. query engine: the retriever returns nodes; the query engine returns an answer.

Creating One


 
The response carries both the answer and the source nodes — which is how you build citations.
 
The Three Components
 
     
  • Retriever — fetches candidate nodes (similarity_top_k).
    •  
  • Node postprocessors — filter, rerank, or expand nodes (e.g. SimilarityPostprocessor, CohereRerank, sentence-window replacement).
    •  
  • Response synthesiser — turns nodes into the final answer.
    •  
 
Response Modes (Synthesis Strategies)
 
How the answer is composed from the nodes:
 
ModeHow it worksUse for
compactStuff as many nodes as fit, then answerDefault — efficient
refineAnswer with node 1, then refine with each nextAccuracy over many nodes
tree_summarizeHierarchically summariseSummarising a corpus
simple_summarizeTruncate + one callSpeed
 
Query Engine Types
 
     
  • Standard — retrieve + synthesise over one index.
    •  
  • Router — an LLM picks the right engine for the question.
    •  
  • Sub-question — decomposes a complex question, queries each part, then combines.
    •  
  • Multi-step — iterative reasoning over several retrieval steps.
    •  
  • SQL / structured — text-to-SQL over databases.
    •  
 
The Router Pattern
 
With several indexes, a router sends the question to the right one.
 

      
        
        Whiteboard
      
      
    
  
 Whiteboard diagram  
 
 
Query Engines as Agent Tools
 
Wrap a query engine as a QueryEngineTool and hand it to an agent — the agent then decides which corpus to query. This is the standard agentic RAG pattern in LlamaIndex: separate engines over separate data partitions, all available as tools.
 
Chat Engines
 
A chat engine is a query engine with conversation memory — use index.as_chat_engine() for multi-turn Q&A instead of one-shot queries.
 
Best Practices
 
     
  • Tune similarity_top_k, then add a reranker postprocessor.
    •  
  • Pick the response mode to fit the task (compact by default).
    •  
  • Use response.source_nodes for citations.
    •  
  • Use a router or sub-question engine for multi-corpus or complex questions.
    •  
 
Summary
 
     
  • A query engine is retrieval + synthesis in one: question → answer with sources.
    •  
  • It contains a retriever, optional postprocessors, and a response synthesiser.
    •  
  • Response modes (compact, refine, tree_summarize) control how the answer is composed.
    •  
  • Router and sub-question engines handle multi-corpus and complex questions.
    •  
  • Wrap engines as tools for agentic RAG, or use a chat engine for multi-turn. EOF echo created