Learning Probabilistic Sentence Representations from Paraphrases
Table 4: Percentage of cases in which hypothesis has larger entropy (or smaller norm for non-probabilistic models) than premise for equal-length sentence pairs in the SNLI test set. In this setting, GloVe and ELMo would give the same results under either sum or avg.
['[EMPTY]', 'Entailment', 'Neutral', 'Contradiction']
Although ELMo can also achieve similar accuracy in the entailment category, it seems to conflate entailment with contradiction, where it shows the highest percentage of all models. Other models, including BERT, GloVe, and Skip-thought, are much closer to random (50%) for entailing pairs.