Jointly Embedding Entities and Text with Distant Supervision
Table 8: MSH WSD disambiguation accuracy. Definitions is comparable to Pakhomov2016, using jointly-embedded words. All differences are significant (McNemar’s test, p≪0.01).
['Method', 'Accuracy %']
We compare against concept embeddings learned by \newciteSabbir2017. In addition, we leverage our jointly-embedded entities and words by adding in the definition-based model used by \newcitePakhomov2016, which calculates an entity’s embedding as the average of definitions of its neighbors in the UMLS hierarchy McInnes et al. This yields a large performance boost of over 6% absolute, indicating that using entities and words together makes up much of the gap between our distantly supervised embeddings and the external resources used by \newciteSabbir2017. Using the definition-based method alone with our jointly-embedded words, we see a significant increase over \newcitePakhomov2016, indicating the benefits of joint training. However, the combined entity and definition model still yields a significantly different 2% boost in accuracy over definitions alone.