Definition
A quantitative metric in RAG evaluation that measures how pertinent a generated response is to the user's initial query, typically calculated by generating potential questions from the answer and measuring their cosine similarity to the original prompt. It ensures the system avoids 'hallucinating' irrelevant information, though optimizing too strictly for it can trade off against conversational nuance or helpful verbosity.
Measures 'topic alignment' with the query, not 'factual accuracy' against the source documents.
"A key perfectly carved to match the internal pins of a specific lock, regardless of what the key is made of."
- Faithfulness(Sibling metric in the RAG Triad (measures grounding in context))
- Cosine Similarity(Mathematical Component)
- LLM-as-a-Judge(Implementation Methodology)
- Context Relevance(Upstream Component (measures quality of retrieved chunks))
Conceptual Overview
A quantitative metric in RAG evaluation that measures how pertinent a generated response is to the user's initial query, typically calculated by generating potential questions from the answer and measuring their cosine similarity to the original prompt. It ensures the system avoids 'hallucinating' irrelevant information, though optimizing too strictly for it can trade off against conversational nuance or helpful verbosity.
Disambiguation
Measures 'topic alignment' with the query, not 'factual accuracy' against the source documents.
Visual Analog
A key perfectly carved to match the internal pins of a specific lock, regardless of what the key is made of.