Can Embeddings Adequately Represent Medical Terminology?New Large-Scale Medical Term Similarity Datasets Have the Answer!Please refer to this version for up-to-date experimental results.
Table D.1: Relative overlap error (O) and average similarity scores between concepts in the same and different categories. For each embedding the best/worst performing similarity metric is reported.
['[EMPTY]', 'metric (best/worst)', '[ITALIC] O (best/worst)', 'sim(DP,TP)', 'sim(DP,Org)', 'sim(TP,TP)', 'sim(DP,DP)', 'sim(Org,Org)']
Interestingly, the best performance of ELMoPubMed is achieved using r, whereas τ – which performed best on the term similarity task – produces the worst results. Furthermore, the MeSH embeddings, performing best on the term similarity task, exhibit comparably bad performance here. These observations provide interesting first insights for future work. We observe that all embeddings result in a higher similarity for the former compared to the latter, as desired. As a comparison, we also compute the average similarity scores between all concept pairs of a category We expect these in-category similarity scores to be higher than any of the cross-category similarity scores, which is indeed the case.