Breaking Sticks and Ambiguities with Adaptive Skip-gram
Table 8: Spearman’s rank correlation results for contextual similarity task on SCWS dataset. Numbers are multiplied with 100.
['[BOLD] MODEL', '[ITALIC] AvgSimC', '[ITALIC] MaxSimC']
NP-MSSG model of Neelakantan et al. This may suggest that SCWS is of limited use for evaluating multi-prototype word representation models as the ability of differentiating between word senses is not necessary to achieve a good score. One may consider another example of an undesirable model which will not be penalized by the target metric in the world similarity task. That is, if a model learned too many prototypes for a word, e.g. with very close vector representations it is hardly usable in practice, but as long as averaged similarities between prototypes correlate with human judgements such non-interpretability will not be accounted during evaluation. We thus consider word-sense induction as a more natural task for evaluation since it explicitly accounts for proper and interpretable mapping from contexts into discovered word meanings.