Ranking Significant Discrepancies in Clinical Reports
Table 1: Ranking performance of our method and baselines.
['Model', 'nDCG@1', 'nDCG@5', 'nDCG', 'P@1', 'P@5', 'R-Prec']
Results. Test set Our optimal model consists of unigram, bi-gram, tri-gram, and RadLex scores. Our method improves R-Prec by 7.4%, nDCG@1 and nDCG@5 performance by 4.5%, and P@5 performance by 4.7%. In 54% of the test cases, our approach improves the nDCG@5 score over SimReg (decreases performance in only 27% of cases). Our model also outperforms leading language model classification approaches (BERT and SciBERT) and a leading neural ranking approach tuned for this task in most metrics (BiPACRR) by up to 15.4% in nDCG@1. We attribute this improved effectiveness of our approach to the explicit modeling of term importance and overlap, which are critical for the task.