Word Embeddings and Stability
Table 1.2: Outline of the seven different Wikipedia corpora used to train word embeddings. The XML Wikipedia dumps that were used in our experiments were created on the 1st of September 2019. The most current ones can be obtained from https://dumps.wikimedia.org/. As per the default settings of our embedding models, only words with five or more occurrences are included in the vocabulary.
['[BOLD] Language', '[BOLD] Shortcut', '[BOLD] Tokens [×106]', '[BOLD] Vocabulary [×104]']
The extrinsic instability Iext(T,C) seems to decrease with corpus size and increase with vocabulary size for word2vec, and fastText. For GloVe, we cannot confirm a correlation based on the data we have.