Word Embeddings and Stability
Table 2.6: Spearman correlation of the metrics dPIP for 1000 target words for different values of |V′|∈{103,104,105,|V|} for word2vec (top), GloVe (middle) and fastText (bottom), obtained as outlined in Section 2.2.3. For each of the techniques, we show the average of the correlation over all languages mentioned in Section 1.2. One might ask why the values on the main diagonal are different from 1: This is the case, since we repeated the random sampling of the proxy words twice and calculated the correlation between these two samples. This helps to understand not only how stable the measures are over different numbers of proxy words, but also over subsequent runs with a fixed number of proxy words.
['[EMPTY]', '103', '104', '105', '|V|']
Hence, our approach to increase the computational efficiency by calculating the word-wise reduced PIP loss over a randomly sampled subset V′ does not interfere with the meaningfulness of the metric.