An Improved Historical Embedding without Alignment
Table 2. Ability to capture semantic similarity.
['[BOLD] Short', '[BOLD] Methods SVD', '[BOLD] Correlation 0.590', '[BOLD] Methods SGNS', '[BOLD] Correlation 0.284']
We tested semantic similarity in both short term and long term. For the short term, each time period covers a year (t=1989, t+1=1990); and for the long term, each time period covers ten years (t=1980-1989, t+1=1990-1999). For each word, the word embeddings based on the corpus segment covering the two time periods are generated using SGNS and SVD; the word embeddings are generated using TSGNS and TSVD where t is the first time period and t+1 is the next time period. For the 3000 words, we test the Spearman’s correlation between the word embedding similarities and human judgments TSGNS and SGNS have comparable performance; TSVD and SVD have comparable performance. The results verify that TSGNS and TSVD can properly measure the semantic similarity at a particular time period, although they are designed for word embedding across time.