Definition
Federated RAG is an architectural pattern that enables an LLM to retrieve context from multiple, heterogeneous, and often decentralized data sources—such as disparate vector databases, SQL warehouses, and external APIs—without centralizing the data. It preserves data sovereignty and avoids massive ETL overhead but introduces challenges regarding latency and the calibration of relevance scores across different indexing systems.
Unlike standard RAG which queries a single unified vector store, Federated RAG orchestrates queries across multiple 'siloed' indexes.
"A lead detective (LLM) calling multiple specialized precinct archives (data silos) simultaneously to piece together a case file."
Conceptual Overview
Federated RAG is an architectural pattern that enables an LLM to retrieve context from multiple, heterogeneous, and often decentralized data sources—such as disparate vector databases, SQL warehouses, and external APIs—without centralizing the data. It preserves data sovereignty and avoids massive ETL overhead but introduces challenges regarding latency and the calibration of relevance scores across different indexing systems.
Disambiguation
Unlike standard RAG which queries a single unified vector store, Federated RAG orchestrates queries across multiple 'siloed' indexes.
Visual Analog
A lead detective (LLM) calling multiple specialized precinct archives (data silos) simultaneously to piece together a case file.