RPD: A Distance Function Between Word Embeddings
Table 1: RPDs of SGNS vs other methods
['[BOLD] Methods', 'GloVe', 'SVDPMI', 'SVDLC']
In our case, we estimate μ=0.953 and σ=0.001 with Monte Carlo simulation with randomly initialized embeddings. Take RPD(ESGNS1, ESVDPMI) = 0.511 Thus, we can confidently reject H0. Notice that we can test any two sets of word embeddings with this method. ’s theory. an interesting phenomenon is that SGNS becomes closer to other methods with the decrease of negative samples, which suggests that negative sampling is one of the factors driving SGNS away from matrix factorization methods. With RPDs between different sets of word embeddings, we plot the embeddings in 2D by treating each embedding space as a single point. We first fix point SVDPMI and SVDLC, then we draw other points according to their RPDs with the other methods. Increasing the number of negative samples pulls SGNS away from SVDPMI.