Definition
Query Expansion is a pre-retrieval technique in RAG pipelines where an LLM reformulates a single user query into multiple variations or semantically enriched versions to improve retrieval recall. While it significantly increases the likelihood of finding relevant context, it introduces an architectural trade-off of increased latency and the risk of retrieving irrelevant 'noise' that can degrade the final generation.
It focuses on broadening the search surface area during the retrieval phase, rather than refining the prompt for the final answer generation.
"Switching from a narrow laser pointer to a wide-beam floodlight to find an object in a dark room."
- HyDE (Hypothetical Document Embeddings)(A specialized expansion technique focusing on synthesized answers.)
- Recall(The primary retrieval metric this technique aims to maximize.)
- Vector Database(The target system where expanded queries are executed.)
- Re-ranking(A downstream component often used to filter the noise introduced by expansion.)
Conceptual Overview
Query Expansion is a pre-retrieval technique in RAG pipelines where an LLM reformulates a single user query into multiple variations or semantically enriched versions to improve retrieval recall. While it significantly increases the likelihood of finding relevant context, it introduces an architectural trade-off of increased latency and the risk of retrieving irrelevant 'noise' that can degrade the final generation.
Disambiguation
It focuses on broadening the search surface area during the retrieval phase, rather than refining the prompt for the final answer generation.
Visual Analog
Switching from a narrow laser pointer to a wide-beam floodlight to find an object in a dark room.