Definition
The mathematical quantification of similarity—typically via cosine similarity, dot product, or BM25—between a query and indexed data to rank context candidates. It involves an architectural trade-off between precision (high thresholds reduce noise but risk missing relevant nuance) and recall (low thresholds provide more context but increase the risk of LLM distraction or hallucinations).
Measures semantic or lexical alignment to a specific query, not the general popularity or static importance of a document.
"A Geiger counter that clicks faster as it approaches the specific 'radioactive' data chunk requested by the user's query."
- Vector Embeddings(Prerequisite)
- Cosine Similarity(Component)
- Cross-Encoders(Component)
- Top-K Retrieval(Component)
Conceptual Overview
The mathematical quantification of similarity—typically via cosine similarity, dot product, or BM25—between a query and indexed data to rank context candidates. It involves an architectural trade-off between precision (high thresholds reduce noise but risk missing relevant nuance) and recall (low thresholds provide more context but increase the risk of LLM distraction or hallucinations).
Disambiguation
Measures semantic or lexical alignment to a specific query, not the general popularity or static importance of a document.
Visual Analog
A Geiger counter that clicks faster as it approaches the specific 'radioactive' data chunk requested by the user's query.