An Improved Historical Embedding without Alignment
Table 3. Average correlation between embedding norm and normalized frequency of selected words.
['[BOLD] Methods PPMI', '[BOLD] Correlation 0.609', '[BOLD] Methods SVD', '[BOLD] Correlation 0.248']
Vector Norm vs. Frequency These word vector norms can be viewed as a time series for detecting the trend concepts behind words with more robustness than word frequency. Here, we test the Spearman’s correlation between the embedding norm of 10 randomly selected words and the normalized frequency of corresponding words along time from 1900 to 1999. Given a word, the normalized frequency is the number of times it appears in a corpus segment normalized by the total number of words in the same corpus segment. We can observe that the TSGNS has word embedding norm more consistent with the normalized frequency than other low-dimensional embedding methods, and has a similar performance as PPMI.