Definition
Mean Reciprocal Rank (MRR) at K is an evaluation metric that calculates the average of the reciprocal ranks of the first relevant document within the top K results. In RAG pipelines, it assesses the retriever's ability to place the 'ground truth' context at the very top of the retrieved list, which is critical because LLMs often suffer from 'lost-in-the-middle' phenomena and perform best when the most relevant data appears first.
Unlike Recall@K which counts how many relevant items were found, MRR cares exclusively about the rank of the first correct one.
"A gold-silver-bronze podium where the score drops sharply from 1 point for 1st place, to 0.5 for 2nd, and 0.33 for 3rd; anything below K is zero."
Conceptual Overview
Mean Reciprocal Rank (MRR) at K is an evaluation metric that calculates the average of the reciprocal ranks of the first relevant document within the top K results. In RAG pipelines, it assesses the retriever's ability to place the 'ground truth' context at the very top of the retrieved list, which is critical because LLMs often suffer from 'lost-in-the-middle' phenomena and perform best when the most relevant data appears first.
Disambiguation
Unlike Recall@K which counts how many relevant items were found, MRR cares exclusively about the rank of the first correct one.
Visual Analog
A gold-silver-bronze podium where the score drops sharply from 1 point for 1st place, to 0.5 for 2nd, and 0.33 for 3rd; anything below K is zero.