Towards UniversalParaphrastic Sentence Embeddings
Table 7: Performance (Pearson’s r×100) as a function of the maximum number of tokens in the sentence pairs over all 20 SemEval STS datasets.
['Max Length', 'paragram-phrase', 'LSTM (no o.g.)', 'LSTM (o.g.)', 'paragram-sl999']
The first hypothesis we test is based on length. Since PPDB contains short text snippets of a few words, the LSTM may not know how to handle the longer sentences that occur in our evaluation tasks. If this is true, the LSTM would perform much better on short text snippets and its performance would degrade as their length increases. To test this hypothesis, we took all 12,108 pairs from the 20 SemEval STS tasks and binned them by length. We then computed the Pearson’s r for each bin.