Definition
Retrieval-Augmented Fine-Tuning (RAFT) is a specialized training methodology that teaches LLMs to ignore irrelevant 'distractor' documents and strictly derive answers from provided relevant context. It involves a trade-off where the model sacrifices some general-purpose zero-shot reasoning for superior high-precision performance in domain-specific RAG pipelines.
Distinct from the Raft consensus algorithm used in distributed systems; RAFT focuses on model alignment for retrieval tasks.
"A gold prospector trained to instantly distinguish between 'fool's gold' (pyrite) and actual gold nuggets within a pan full of river silt."
- RAG(Deployment Context)
- Supervised Fine-Tuning (SFT)(Prerequisite)
- Distractor Documents(Component)
- Chain-of-Thought (CoT)(Component)
Conceptual Overview
Retrieval-Augmented Fine-Tuning (RAFT) is a specialized training methodology that teaches LLMs to ignore irrelevant 'distractor' documents and strictly derive answers from provided relevant context. It involves a trade-off where the model sacrifices some general-purpose zero-shot reasoning for superior high-precision performance in domain-specific RAG pipelines.
Disambiguation
Distinct from the Raft consensus algorithm used in distributed systems; RAFT focuses on model alignment for retrieval tasks.
Visual Analog
A gold prospector trained to instantly distinguish between 'fool's gold' (pyrite) and actual gold nuggets within a pan full of river silt.