Post-Processing of Word Representations via Variance Normalization and Dynamic Embedding
Table 3: The SRCC performance comparison (×100) for SGNS alone, SGNS+PPA and SGNS+PVN against word analogy datasets.
['Type: Google', 'Type: Add', 'SGNS 59.6', 'PPA 61.3', 'PVN(ours)  [BOLD] 62.1']
We use addition as well as multiplication evaluation methods. PVN performs better than PPA in both. For the multiplication evaluation, the performance of PPA is worse than the baseline. In contrast, the proposed PVN method has no negative effect as it performs consistently well. This can be explained below. When the multiplication evaluation is adopted, the missing dimensions of the PPA influence the relative angles of vectors a lot. This is further verified by the fact that some linguistic properties are captured by these high-variance dimensions and their total elimination is sub-optimal.