Closed Form Word Embedding Alignment
Table 6: Similarity and Analogy tests before and after alignment and combining embeddings derived from different techniques and datasets by AO+Scaling. Best scores in bold.
['TestSets', 'G(W)', 'W(W)', '[G(W)⊙W(W)]', 'W(GN)', '[G(W)⊙W(GN)]', 'G(CC840)', '[G(CC840)⊙W(GN)]']
The first set shows the default Wikipedia data set under GloVe (G(W)), under word2vec (W(W)), and combined ([G(W)⊙W(W)]). The second set shows word2vec embedding of GoogleNews (W(GN)), and combined ([G(W)⊙W(GN)]) with G(W). The third set shows GloVe embedding of CommonCrawl (840B) (G(CC840)) and then combined with W(GN) as [G(CC840)⊙W(GN)]. Combining two embeddings consistently boosts the performance on similarity and analogy tests. The best score on each experiment is in bold, and in 5 out of 6 cases, it is from a combined embedding. Indeed, in each experiment, except one case (G(CC840) on the SEM analogies), the combined result is always better than either of the originals.