Word Embeddings for Entity-annotated Texts
Table 2: Word analogy results. Shown are the prediction accuracy for the normal analogy tasks and the variation in which predictions are limited to the correct entity type. The best two values per task and variation are highlighted.
['[EMPTY]', 'normal analogy  [ITALIC] rW2V', 'normal analogy  [ITALIC] rGLV', 'normal analogy  [ITALIC] aW2V', 'normal analogy  [ITALIC] aGLV', 'normal analogy DW [ITALIC] id', 'normal analogy DW [ITALIC] log', 'normal analogy VRS', 'typed analogy  [ITALIC] aW2V', 'typed analogy  [ITALIC] aGLV', 'typed analogy DW [ITALIC] id', 'typed analogy DW [ITALIC] log', 'typed analogy VRS']
We attribute this to the size of the data set that contains less than the billions of tokens that are typically used to train for this task. GloVe performs better than word2vec for this task on both raw and annotated data, while VERSE does best overall. The typed task, in which we also provide the entity type of the target word, is easier and results in better scores. If we consider only the subset of 6,892 location targets for the GA task, we find that the graph-based models perform much better, with VERSE being able to predict up to 1,662 (24.1%) of location targets on its best run, while aW2V and aGLV are only able to predict 14 (0.20%) and 16 (0.23%), respectively. For this entity-centric subtask, VERSE is clearly better suited. For the MSR task, which does not contain entities, we do not observe such an advantage.