Word Embeddings and Stability
Table 1.6: Coverage of our models on the word analogy tasks for different languages compared to the results published by grave-etal-2018-learning.
['[EMPTY]', 'Hi', 'Fi', 'Zh', 'Cs', 'Pl', 'Pt', 'En']
Following grave-etal-2018-learning, the vocabulary of each model was restricted to the 200,000 most frequent words from the training data before evaluating the model on the word analogy dataset. This means, that a fraction of the questions of the analogy tasks is not answered, as they contain out-of-vocabulary words. Therefore, to compare the reported scores one also needs to take the coverage, i.e. the percentage of answered questions, into account. bojanowski2016 did not publish these numbers.