An Improved Historical Embedding without Alignment
Table 4. Ability to capture the attested shifts.
['[BOLD] Short', '[BOLD] Methods PPMI', '[BOLD] Correlation 0.423', '[BOLD] Methods DW2V', '[BOLD] Correlation']
Clearly, TSGNS beats SGNS and SVD, and has similar performance as PPMI. It is worth pointing out that PPMI does not enjoy the advantages of low-dimensional embeddings such as higher efficiency and better generalization. In specific, the low-dimensional embeddings are influenced more or less by the context change of all words over time; on the other hand, the high-dimensional embeddings using PPMI are isolated from word to word, i.e., the embedding of a word is not influenced by the context change of other words over time. It helps explain why PPMI performs even worse in long term since it is expected that context change in long term is greater than in short term.