All-but-the-Top: Simple and Effective Postprocessing for Word Representations
Table 19: Before-After results (x100) on the word analogy task.
['[EMPTY]', 'WORD2VEC orig.', 'WORD2VEC proc.', 'GLOVE orig.', 'GLOVE proc.']
It can be noticed that while postprocessing continues to improve the performance, the improvement is not as pronounced as earlier. We hypothesize that this is because the mean and some dominant components get canceled during the subtraction of v(w2) from v(w1), and therefore the effect of postprocessing is less relevant.