Passage Re-ranking: A Comparative Analysis
Table 4. Average MUR with respect to the different cut-off values (i).
['[BOLD] i', '1', '2', '3', '4', '5', '6', '7', '8', '9', '10']
Similarity between the BERT’s and the BM25’s result lists. To measure the similarity between the results of BERT and BM25, we calculate the following metric, MUR - matches upto result. MUR(i,q) for a query q measures the number of matches in the top i results of BERT and BM25. We can see the average MUR for each i∈[1,10] The number of matches increases linearly with i with slope of about 0.33 and intercept around -0.21, indicating a consistent linear relationship between BERT and BM25.