Linear Algebraic Structure of Word Senses, with Applications to Polysemy
Table 3: The average relative errors and cosine similarities between the vectors of pseudowords and those predicted by Theorem 2. m pairs of words are randomly selected and for each pair, all occurrences of the two words in the corpus is replaced by a pseudoword. Then train the vectors for the pseudowords on the new corpus.
['[ITALIC] m pairs relative error', '[ITALIC] m pairs SN', '10 0.32', '103 0.63', '3⋅104 0.67']
When m is small, the error is small and the cosine similarity is as large as 0.9. Even if m=3⋅104 (i.e., about 90% of the words in the vocabulary are replaced by pseudowords), the cosine similarity remains above 0.7, which is significant in the 300 dimensional space. This provides positive support for our analysis.