Word Embeddings and Stability
Table 1.4: Scores of our fastText models (skip-gram) on the word analogy tasks for different languages compared to the results published by bojanowski2016 and grave-etal-2018-learning. The results of this work, noted in the rightmost column, state the mean μ, standard deviation σ, and highest score of 128 runs on independently shuffled corpora.
['[BOLD] Language', '[BOLD] Bojanowski (2016)', '[BOLD] Grave (2018)', '[BOLD] This Work  [ITALIC] μ', '[BOLD] This Work  [ITALIC] σ', '[BOLD] This Work Best']
Following grave-etal-2018-learning, the vocabulary of each model was restricted to the 200,000 most frequent words from the training data before evaluating the model on the word analogy dataset. This means, that a fraction of the questions of the analogy tasks is not answered, as they contain out-of-vocabulary words. Therefore, to compare the reported scores one also needs to take the coverage, i.e. the percentage of answered questions, into account. bojanowski2016 did not publish these numbers.