Linear Algebraic Structure of Word Senses, with Applications to Polysemy
Table 4: The average cosine of the angles between the vectors of words and the span of vector representations of its senses. The words tested are those in the WSI task of SemEval 2010.
['vector type', 'GloVe', 'skip-gram', 'SN']
Test 2. The second test is a proxy for what would be a complete (but laborious) test of the Linearity Assertion: replicating the thought experiment while hand-labeling sense usage for many words in a corpus. The simpler proxy is as follows. For each word w, WordNet The text embedding for sense s should approximate the ground truth vector vs for it. Then the Linearity Assertion predicts that embedding vw lies close to the subspace spanned by the sense vectors. (Note that this is a nontrivial event: in 300 dimensions a random vector will be quite far from the subspace spanned by some 3 other random vectors.) We tested three standard word embedding methods: The results show that the word vectors are quite close to the subspace spanned by the senses.