All-but-the-Top: Simple and Effective Postprocessing for Word Representations
Table 20: Before-After results (x100) on the semantic textual similarity tasks.
['[EMPTY]', 'WORD2VEC orig.', 'WORD2VEC proc.', 'GLOVE orig.', 'GLOVE proc.']
Representing sentences by the average of their constituent word representations is surprisingly effective in encoding the semantic information of sentences ( We follow this rubric and represent a sentence s based on its averaged word representation, i.e., v(s)=1|s|∑w∈sv(w), and then compute the similarity between two sentences via the cosine similarity between the two representations.