GLOSS: Generative Latent Optimization of Sentence Representations
Table 1: Test performance on unsupervised STS-(12-16, B) and supervised tasks. (α) indicate results computed by us. (β) PV-DBOW results are taken from Pagliardini et al. (2017). (†) The unsupervised results for Skip-thought are taken from Arora et al. (2017) and the supervised ones from Pagliardini et al. (2017). (∗) Following SentEval, STS-13 does not include the SMT dataset due to licensing issues. The STS-B scores of other baselines are from the official webpage, except for InferSent which are from Wieting and Gimpel (2018). All other results are from the respective publications. See Table 3 in supplementary material for results with higher dimensionality.
['Model', 'Config. #Tok', 'Config. Dim', 'Unsupervised STS-() tasks 12', 'Unsupervised STS-() tasks 13*', 'Unsupervised STS-() tasks 14', 'Unsupervised STS-() tasks 15', 'Unsupervised STS-() tasks 16', 'Unsupervised STS-() tasks B', 'Unsupervised STS-() tasks Avg', 'Supervised tasks MR', 'Supervised tasks CR', 'Supervised tasks SUBJ', 'Supervised tasks MPQA', 'Supervised tasks TREC', 'Supervised tasks Avg']
GLOSS-BoW performs particularly well on STS-Benchmark where it outperforms all methods. GLOSS-BoW is competitive to Sent2Vec, even though the latter was trained on 30 times more training data. In fact, it also matches the performance of InferSent which requires labeled training data. This shows that GLOSS is very data efficient which makes it attractive for low-resource languages. Amongst unsupervised methods, GLOSS-POS is slightly behind Sent2Vec which was trained on much more data. Both GLOSS-POS and GLOSS-BoW outperform paragraph vectors (PV-DBOW). Skip-thought and InferSent use much larger vectors and they require corpora with ordered sentences. We expect that increasing the dimensionality of z even further would also help our models (cf. However, we show that strong performance can be achieved with reasonable representation sizes.