Definition
ColBERT is a retrieval model that employs a 'late interaction' mechanism, encoding queries and documents into sets of token-level embeddings and delaying their interaction until the retrieval stage via a MaxSim operation. This architecture allows for more granular matching than standard bi-encoders while maintaining significantly lower latency than traditional cross-encoders.
Unlike standard dense retrieval which reduces a document to a single vector, ColBERT stores multiple vectors per document to preserve token-level semantic information.
"The Multi-Point Laser Array: Instead of a single flashlight beam (bi-encoder) trying to illuminate a whole room, it uses a grid of lasers to find exact point-to-point matches between the query and the text."
- Late Interaction(Core Mechanism)
- MaxSim(Scoring Operation)
- Bi-Encoder(Architectural Alternative)
- BEIR(Evaluation Benchmark)
- Cross-Encoder(Performance Standard)
Conceptual Overview
ColBERT is a retrieval model that employs a 'late interaction' mechanism, encoding queries and documents into sets of token-level embeddings and delaying their interaction until the retrieval stage via a MaxSim operation. This architecture allows for more granular matching than standard bi-encoders while maintaining significantly lower latency than traditional cross-encoders.
Disambiguation
Unlike standard dense retrieval which reduces a document to a single vector, ColBERT stores multiple vectors per document to preserve token-level semantic information.
Visual Analog
The Multi-Point Laser Array: Instead of a single flashlight beam (bi-encoder) trying to illuminate a whole room, it uses a grid of lasers to find exact point-to-point matches between the query and the text.