Closed Form Word Embedding Alignment
Table 4: Spearman coefficient scores for synonym and analogy tests between the aligned word2vec to GloVe embeddings, GloVe transformed to word2vec and between GloVe embeddings of Wikipedia and CC42 dataset.
['TestSets', 'GloVe', 'word2vec', 'GloVe to word2vec untransformed', 'GloVe to word2vec + rotation', 'GloVe to word2vec + scaling', 'GloVe to word2vec + translation', 'GloVe to word2vec normalized']
Before transformation (“untransformed”) these across-dataset comparison is very poor, close to 0. Alignment with just AO-Rotation achieves scores nearly as good as, and sometimes better than on the original datasets. word2vec scores higher than GloVe, and the across-dataset scores are typically between these two scores. Adding scaling, with AO+Scaling and also translation have small positive affect. We also consider first normalizing all vectors ai∈A and bi∈B so ∥ai∥=∥bi∥=1 before alignment, as it would optimize average cosine similarity. This “normalized” score performs slightly worse, probably because it deemphasizes the alignment of more stably embedded vectors.