Definition
The practice of leveraging the weights and reasoning capabilities of a pre-trained foundation model—trained on massive general datasets—and repurposing it as the core engine for specific RAG or Agentic tasks. It allows developers to achieve high performance on niche domains with minimal additional training, though it presents a trade-off between the efficiency of general knowledge and the potential for 'catastrophic forgetting' or bias transfer from the source model.
Distinguish from RAG's external data retrieval; transfer learning is about the model's internal 'intelligence' architecture, not its external library.
"A master chef (the pre-trained model) being hired to work in a specific sushi kitchen (the target domain): they already know how to use knives and heat, they just need to learn the specific recipes."
- Foundation Model(Prerequisite)
- Fine-Tuning(Component)
- Domain Adaptation(Component)
- In-Context Learning(Component)
Conceptual Overview
The practice of leveraging the weights and reasoning capabilities of a pre-trained foundation model—trained on massive general datasets—and repurposing it as the core engine for specific RAG or Agentic tasks. It allows developers to achieve high performance on niche domains with minimal additional training, though it presents a trade-off between the efficiency of general knowledge and the potential for 'catastrophic forgetting' or bias transfer from the source model.
Disambiguation
Distinguish from RAG's external data retrieval; transfer learning is about the model's internal 'intelligence' architecture, not its external library.
Visual Analog
A master chef (the pre-trained model) being hired to work in a specific sushi kitchen (the target domain): they already know how to use knives and heat, they just need to learn the specific recipes.