Linear Ensembles of Word Embedding Models
Table 2: Average mean ranks of the synonym test, smaller values are better. The best result in each row is in bold. All differences are statistically significant: with p<2.2⋅10−16 for all cases.
['[BOLD] Dim', '[BOLD] SOLS', '[BOLD] SOPP', '[BOLD] Mean  [ITALIC] W']
SOPP, on the other hand, produces a combined model where the synonym similarities are ranked higher than in initial models. This means that the SOPP combined models pull the synonymous words closer together than they were in the initial models. The differences in mean ranks were tested using paired Wilcoxon signed-rank test at 95% confidence level and the differences were statistically significant with p-value being less than 2.2⋅10−16 in all cases. In overall, the SOPP ranks are on average 10% lower than the mean ranks of the initial models. The absolute improvement on average between SOPP and mean of W is 3476. Although the first bin covering the rank range from 1 to 10 contains the most words for both models and the number of synonym pairs falling to further rank bins decreases the curve is not steep and close to 100 words (87 in case of SOPP and 94 in case of the initial model) belong to the last bin counting ranks higher than 100000. Looking at the farthest synonym pairs revealed that one word in these pairs is typically polysemous and its sense in the synonym pair is a relatively rarely used sense of this word, while there are other more common senses of this word with a completely different meaning. We give some examples of such synonym pairs: kaks (two) - puudulik (insufficient): the sense of this pair is the insufficient grade in high school, while the most common sense of the word kaks is the number two; ida (east) - ost (loan word from German also meaning east): the most common sense of the word ost is purchase; rubla (rouble) - kull (bank note in slang): the most common sense of the word kull is hawk.