Model Comparison for Semantic Grouping
Table 1: Comparison of Spearman correlations on the STS datasets between the two similarity measures we introduce in the text. The average is weighted according to dataset size.
['Embedding', 'Method', 'STS12', 'STS13', 'STS14', 'STS15', 'STS16', 'Average']
We note that the Gaussian is a much better modelling choice, beating the vMF on every dataset by a margin of at least 0.05 (5%) on average, with each of the three word embeddings. This is strong evidence that the information encoded in an embedding’s magnitude is useful for tasks such as semantic similarity.