Definition
Big O Notation quantifies the computational complexity and scalability of RAG pipeline components, specifically measuring how time or memory requirements grow relative to the size of the vector database (N) or the input context length (L). It is essential for evaluating the efficiency of approximate nearest neighbor (ANN) search algorithms and the quadratic cost of self-attention mechanisms in LLMs.
Measures asymptotic scalability of retrieval and inference, not millisecond latency.
"The Librarian’s Sorting Strategy: Comparing the effort of checking every single book in a library (O(N)) versus using an optimized card catalog (O(log N))."
- HNSW (Hierarchical Navigable Small World)(Implementation)
- Self-Attention Mechanism(Complexity Constraint)
- Vector Indexing(Prerequisite)
- Brute-force Search (Flat Index)(Baseline Comparison)
Conceptual Overview
Big O Notation quantifies the computational complexity and scalability of RAG pipeline components, specifically measuring how time or memory requirements grow relative to the size of the vector database (N) or the input context length (L). It is essential for evaluating the efficiency of approximate nearest neighbor (ANN) search algorithms and the quadratic cost of self-attention mechanisms in LLMs.
Disambiguation
Measures asymptotic scalability of retrieval and inference, not millisecond latency.
Visual Analog
The Librarian’s Sorting Strategy: Comparing the effort of checking every single book in a library (O(N)) versus using an optimized card catalog (O(log N)).