Definition
The architectural approach of updating a RAG knowledge base or an agent's internal model by processing new data streams sequentially without re-indexing the entire corpus or retraining from scratch. This method optimizes for low latency and resource efficiency while managing the trade-off between immediate data availability and potential index fragmentation or 'embedding drift'.
In RAG, this refers to 'upserting' new documents into a vector store rather than performing a full batch re-index.
"A loose-leaf binder where new pages are inserted into specific sections as information arrives, rather than reprinting the entire textbook for every update."
Conceptual Overview
The architectural approach of updating a RAG knowledge base or an agent's internal model by processing new data streams sequentially without re-indexing the entire corpus or retraining from scratch. This method optimizes for low latency and resource efficiency while managing the trade-off between immediate data availability and potential index fragmentation or 'embedding drift'.
Disambiguation
In RAG, this refers to 'upserting' new documents into a vector store rather than performing a full batch re-index.
Visual Analog
A loose-leaf binder where new pages are inserted into specific sections as information arrives, rather than reprinting the entire textbook for every update.