Revisiting Recurrent Networks for Paraphrastic Sentence Embeddings
Table 8: Impact of initializing and regularizing toward universal models (Pearson’s r×100) in supervised training.
['Model', 'Regularization', 'STS', 'SICK']
Initializing and regularizing to the pretrained models significantly improves the performance for all three models, justifying our claim that these models serve a dual purpose: they can be used a black box semantic similarity function, and they possess rich knowledge that can be used to improve the performance of downstream tasks.