Learning Conceptual-Contextual Embeddings for Medical Text
Table 3: Performance on language-inferable and non-language-inferable knowledge
['[EMPTY]', '[BOLD] # of examples', '[BOLD] Hits@10 (%)']
Break-down analysis To measure the effect of semantic generalizability on model performance, and to understand the performance gap between the CC model and TransE , we first sampled 100 examples from the testing set, and labeled them to two categories: Language-inferable (LI) and Non-language-inferable (Non-LI), following our previous definition. First we observe that 3/4 of the triplets contain knowledge that can be inferred from text. This shows that in medical knowledge graphs, a majority of structured knowledge can potentially be carried by text representations. Making use of concept names can be difference-making in medical knowledge embeddings. In the case of the CC model, on LI type examples it gets to 77 percent hit at top10, while for Non-LI type the performance is much lower and is on par with the TransE model.