Model Comparison for Semantic Grouping
Table 2: Comparison of Spearman correlations on the STS datasets between our best model (Diag+AIC) and SIF, WMD and MWV for three different word vectors. The average is weighted according to dataset size.
['Embedding', 'Method', 'STS12', 'STS13', 'STS14', 'STS15', 'STS16', 'Average']
The method proposed in this work is able to out-perform the standard weighting induced by MWV, as well as the WMD approach on all datasets, with each of the three word embeddings considered. We outperform SIF using the GloVe embeddings by 0.0309 (3.09%) and effectively tie when using the FastText and Word2VecGN embeddings with a difference of 0.0057 (0.57%) and 0.0030 (0.30%) respectively. We show that SIF and our method are on par with each other and that both significantly outperform MWV and WMD when using the glove embeddings.