Geometry of Polysemy
Table 6: Performance (Spearman’s rank correlation x100) on SCWS task..
['Model', 'SCWS SoftSim', 'SCWS HardSim', 'SCWS-lite SoftSim', 'SCWS-lite HardSim']
Here we conclude that in general our lexeme representations have a similar performance as the state-of-the-art on both soft and hard decisions. It is worth mentioning that the vanilla word2vec representation (which simply ignores the provided contexts) also has a comparable performance – this makes some sense since some of the words in SCWS are monosemous (and hence their meanings are context-dependent). A closer inspection of the results shows that the vanilla word2vec representation also performs fairly well for polysemous words too – we hypothesize that this is because the word2vec global representation is actualy close to the individual sense representations, if the senses occur frequently enough in the corpus. Since (a) other senses are irrelevant to w′, we can assume vk(w)v(w′)≈0 for k≠k′, and (b) the k′-th sense is frequent enough, we can assume αk′≈1. Putting (a) and (b) together we can conclude that v(w)Tv(w′)≈vk′(w)Tv(w′) and therefore the inner product between v(w) and v(w′) captures the similarity between k′-th sense of w and w′. To separate out the effect of the combination of monosemous and polysemous words in the SCWS dataset , we expurgate monosemous words from the corpus creating a smaller version that we denoted by SCWS-lite. In SCWS-lite, we only consider the pairs of sentences where both target words in one pair have the same surface form but different contexts (and hence potentially different senses). SCWS-lite now contains 241 pairs of words, which is roughly 12% of the original dataset.