\method: Improving Medical Entity Linking with Semantic Type Prediction
Table 3: Evaluating the effect of using WikiMed and PubMedDS for entity disambiguation. Here, we report area under PR-curve for comparing models. T ← X denotes first trained on X dataset then fine-tuning using T. We find that datasets WikiMed and PubMedDS provide a substantial improvement on the task. More details are provided in Section 7.2.
['[EMPTY]', '[BOLD] NCBI', '[BOLD] Bio CDR', '[BOLD] ShARe', '[BOLD] MedMent.']
In this section, we analyze the impact of using our proposed datasets: WikiMed and PubMedDS for entity disambiguation. For this, we compare the performance of \method trained on different datasets. We use area under Precision-Recall curve (AUC) as our evaluation metric. In our results, Training data (T) denotes \method trained using training split of the corresponding dataset. The effect of WikiMed and PubMedDS is analyzed by first using them for training \method and then fine-tuning the model on the training split. This is denoted by T←{WikiMed}{} and T←{PubMedDS}{} respectively. Finally, T←Both denotes the combined model which utilizes both the datasets. T←Both concatenates BERT encoding from T←{WikiMed}{} and T←{PubMedDS}{} models and passing it to a classifier which is trained using the training dataset. We find that there is a substantial gain in performance on using WikiMed and PubMedDS along with the training data. Further, we find that the combined model which allows to incorporate the benefits from both the corpora gives the best performance on all dataset. This shows that both the datasets contain complimentary information relevant for entity disambiguation. Overall, we get an average absolute increase of 6.7, 6.6, and 9.4 AUC from WikiMed, PubMedDS, and the combined model respectively.