Definition
Faithfulness is a metric used to evaluate the groundedness of a generated response by calculating the ratio of claims in the output that can be directly inferred from the retrieved context. It serves as the primary technical guardrail against hallucinations in RAG pipelines, often requiring an LLM-as-a-judge to perform claim extraction and verification.
Unlike 'General Accuracy', Faithfulness only cares if the answer is supported by the specific provided documents, not the model's pre-trained internal knowledge.
"An Auditor's Checklist: Every claim made in the final report must have a corresponding 'receipt' or source citation in the evidence folder."
- Hallucination(Inverse Failure Mode)
- RAG Triad(Conceptual Framework)
- NLI (Natural Language Inference)(Underlying Computational Method)
Conceptual Overview
Faithfulness is a metric used to evaluate the groundedness of a generated response by calculating the ratio of claims in the output that can be directly inferred from the retrieved context. It serves as the primary technical guardrail against hallucinations in RAG pipelines, often requiring an LLM-as-a-judge to perform claim extraction and verification.
Disambiguation
Unlike 'General Accuracy', Faithfulness only cares if the answer is supported by the specific provided documents, not the model's pre-trained internal knowledge.
Visual Analog
An Auditor's Checklist: Every claim made in the final report must have a corresponding 'receipt' or source citation in the evidence folder.