Definition
A transformer-based encoder architecture used in RAG for semantic search and relevance scoring; it utilizes bidirectional self-attention to generate context-aware embeddings, though it faces a computational trade-off between the high speed of Bi-Encoders for initial retrieval and the high precision of Cross-Encoders for re-ranking.
Encoder-only architecture used for 'reading' and 'mapping' data, as opposed to GPT's Decoder-only architecture used for 'writing' text.
"A 360-degree panoramic camera that captures the entire context of a sentence simultaneously to map its exact location in a multidimensional library."
- Transformer(Prerequisite)
- Dense Vector Embeddings(Component)
- Cross-Encoder(Implementation Pattern)
- Bi-Encoder(Implementation Pattern)
Conceptual Overview
A transformer-based encoder architecture used in RAG for semantic search and relevance scoring; it utilizes bidirectional self-attention to generate context-aware embeddings, though it faces a computational trade-off between the high speed of Bi-Encoders for initial retrieval and the high precision of Cross-Encoders for re-ranking.
Disambiguation
Encoder-only architecture used for 'reading' and 'mapping' data, as opposed to GPT's Decoder-only architecture used for 'writing' text.
Visual Analog
A 360-degree panoramic camera that captures the entire context of a sentence simultaneously to map its exact location in a multidimensional library.