An Improved Historical Embedding without Alignment
Table 1. Cosine similarity of word embeddings on selected words.
['[EMPTY]', '[BOLD] Context', '[BOLD] mirage', '[BOLD] localized', '[BOLD] lord', '[BOLD] recollections', '[BOLD] tired', '[BOLD] thunder', '[BOLD] reporter', '[BOLD] bridge', '[BOLD] web', '[BOLD] reject']
At different settings of α, TSGNS is trained using the corpus segments in t and t+1. For each input word, the word embeddings in the two time periods are generated and their cosine similarity is computed. Ten different words are selected. For each word in time period t=1989 and t+1=1990, the cosine similarities at different settings of α are computed using PPMI, SGNS, SVD, DW2V, TSGNS, and TSVD respectively.