Word Embeddings for Entity-annotated Texts
Table 1: Word similarity results. Shown are the Pearson correlations between the cosine similarity of the embeddings and the human score on the word similarity datasets. The two best values per task are highlighted.
['[EMPTY]', '[ITALIC] rW2V', '[ITALIC] rGLV', '[ITALIC] aW2V', '[ITALIC] aGLV', 'DW [ITALIC] id', 'DW [ITALIC] log', 'VRS']
Unsurprisingly, we find that no single model performs best for all tasks. The performance of both methods degrades slightly but consistently when they are trained on the annotated data in comparison to the raw data. The DeepWalk-based models perform better than GloVe but do poorly overall. VERSE performs very well for some of the tasks, but is worse than word2vec trained on the raw data for rare words and the SimLex data. This is likely caused by the conceptual structure of the cooccurrence graph on which VERSE is trained, which captures relatedness and not similarity as tested by SimLex. For the purely term-based tasks in this evaluation that do not contain entity relations, word2vec is thus clearly the best choice for similarity tasks, while VERSE does well on relatedness tasks.