Definition
ColBERT (Contextualized Late Interaction over BERT) is a retrieval model that utilizes a late interaction architecture to encode queries and documents into multi-vector representations at the token level. It enables high-precision retrieval in RAG pipelines by calculating the MaxSim (maximum similarity) between token sets, capturing fine-grained semantic nuances that single-vector bi-encoders often lose.
Unlike standard bi-encoders that produce one vector per document, ColBERT produces a matrix of vectors per document.
"A transparent overlay of two star maps where every individual star is checked for a match, rather than just comparing the center points of the galaxies."
Conceptual Overview
ColBERT (Contextualized Late Interaction over BERT) is a retrieval model that utilizes a late interaction architecture to encode queries and documents into multi-vector representations at the token level. It enables high-precision retrieval in RAG pipelines by calculating the MaxSim (maximum similarity) between token sets, capturing fine-grained semantic nuances that single-vector bi-encoders often lose.
Disambiguation
Unlike standard bi-encoders that produce one vector per document, ColBERT produces a matrix of vectors per document.
Visual Analog
A transparent overlay of two star maps where every individual star is checked for a match, rather than just comparing the center points of the galaxies.