All-but-the-Top: Simple and Effective Postprocessing for Word Representations
Table 3: Before-After results (x100) on word similarity task on seven datasets.
['[EMPTY]', 'WORD2VEC orig.', 'WORD2VEC proc.', 'GLOVE orig.', 'GLOVE proc.']
In our experiment, the algorithm scores the similarity between two words by the cosine similarity between the two corresponding word vectors (CosSim(v1,v2)=v⊤1v2/∥v1∥∥v2∥). These statistics (average improvement of 2.3%) suggest that by removing the common parts, the remaining word representations are able to capture stronger semantic relatedness/similarity between words.