Learning Semantic Textual Similarity from Conversations
Table 2: Performance on SNLI classification of the Reddit+SNLI model with the transformer encoder.
['[EMPTY]', 'Accuracy']
The multitask model achieves 84.1% accuracy and is close to the performance of InferSent. There are two significant differences between our model and prior work. First, the proposed model learns all model parameters from scratch, including the word embeddings. InferSent uses large pre-trained word embeddings which could result in less out of vocabulary words. In our work, the Reddit dataset is large enough that we do not need to use pre-trained word embeddings. Secondly, our multi-task model learns two tasks simultaneously, balancing performance between them, while InferSent only optimizes performance on SNLI. As will be presented below, our multi-task model performs better on STS. We suspect the Reddit data acts as a regularizer to prevent the resulting sentence embedding from overfitting to the SNLI task, thus improving transfer performance on other tasks.