Word Embeddings and Stability
Table 2.3: Spearman correlation of the metrics p@n and j@n for 1000 target words for different values of n∈{2,5,10,25,50} for word2vec, GloVe, and fastText, obtained as outlined in Section 2.2.2. For each of the techniques, we show the average of the correlation for all languages mentioned in Section 1.2.
['[EMPTY]', '[ITALIC] p@2', '[ITALIC] p@5', '[ITALIC] p@10', '[ITALIC] p@25', '[ITALIC] p@50']
If p@n and j@n were to be considered consistent over the free parameter n, we would expect that a target word, which is identified as comparably stable based on n=5, also ranks among the more stable words for n=50. We denote the evaluation of the two metrics p@n and j@n on the same target words, for this set of runs, as p@n–––– and j@n–––. All correlation values are higher than 0.92, which shows that the chosen sample size of 16 runs is indeed sufficient to conclude the underlying distribution.