Towards Learning Cross-Modal Perception-Trace Models
Table 1: Spearman’s rank correlation averaged over all evaluation data sets.
['[BOLD] Embeddings', '[BOLD] Weighted average']
We computed the Spearman’s average rank correlation over all evaluation data sets weighted according to their respective size and obtained the score of 0.451 and 0.565 for GloVe and word2vec representations, respectively. Our embeddings seems to match the human notion of similarity considerably better, by achieving the rank correlation of 0.755 All embeddings show comparable performance on the subsets of MEN and WS-353, word2vec representations being less correlated with the scores provided in the WS-353 subset. The difference in the weighted results lies mainly in the worse performance of the word2vec and GloVe vectors on the subset of SimLex-999. A possible explanation of this gap is that the evaluation data sets MEN and WS-353 reflect human judgments of the relatedness of the word pairs, SimLex-999 explicitly measures concept similarity [bruni2014multimodal, finkelstein2002placing, hill2015simlex]. Although relatedness and similarity are highly connected concepts, strongly associated entities are not necessarily similar. More precisely the pairwise correlation between the ratings in the SimLex-999 subset and the GloVe vectors is negative, word2vec and our model score on the Spearson’s rank correlation 0.205 and 0.462, respectively.