Definition
NDCG (Normalized Discounted Cumulative Gain) is a ranking metric used in RAG to evaluate the retrieval component's ability to order context by relevance, rewarding the placement of highly relevant documents at the top while logarithmically penalizing relevant items that appear lower in the results. It is essential for minimizing 'lost-in-the-middle' phenomena in LLMs by ensuring the highest-signal data is prioritized.
Focuses on the position and graded relevance of results, whereas Precision@K only measures if a result is present.
"A luxury restaurant menu where the signature dishes must be on the first page; if the best items are buried on page ten, the menu's effectiveness score drops significantly."
- Mean Reciprocal Rank (MRR)(Alternative ranking metric)
- Cross-Encoder(Mechanism used to optimize NDCG via re-ranking)
- Cumulative Gain(Mathematical precursor)
Conceptual Overview
NDCG (Normalized Discounted Cumulative Gain) is a ranking metric used in RAG to evaluate the retrieval component's ability to order context by relevance, rewarding the placement of highly relevant documents at the top while logarithmically penalizing relevant items that appear lower in the results. It is essential for minimizing 'lost-in-the-middle' phenomena in LLMs by ensuring the highest-signal data is prioritized.
Disambiguation
Focuses on the position and graded relevance of results, whereas Precision@K only measures if a result is present.
Visual Analog
A luxury restaurant menu where the signature dishes must be on the first page; if the best items are buried on page ten, the menu's effectiveness score drops significantly.