Post-Processing of Word Representations via Variance Normalization and Dynamic Embedding
Table 5: The SRCC performance comparison (×100) for SGNS alone and SGNS+PDE against word similarity and analogy datasets.
['Type', 'SGNS', 'PVN(ours)']
We adopt the same setting in evaluating the PDE method such as the window size, vocabulary size, number of negative samples, training data, etc. when it is applied to SGNS baseline. The final word representation is composed by two parts: ~v(w)=[vs(w)T,vd(w)T]T, where vs(w) is the static part obtained from dimension reduction using PCA and vd(w)=ATv(w) is the projection of v(w) to dynamic subspace A. Here, we set the dimensions of vs(w) and vd(w) to 240 and 60, respectively. By adding the ordered information via PDE, we see that the quality of word representations is improved in both evaluation tasks.