Uncertain Natural Language Inference
Table 4: Metrics for training on u-SNLI.
['[EMPTY]', '[BOLD] Hyp-only  [BOLD] Dev', '[BOLD] Hyp-only  [BOLD] Test', '[BOLD] Full-model  [BOLD] Dev', '[BOLD] Full-model  [BOLD] Test']
Just training on 55,517 u-SNLI examples yields a 62.71% Pearson r on test. The hypothesis-only baseline achieved a correlation around 40%. This result corroborates the findings that a hidden bias exists in the SNLI dataset ’s hypotheses, and shows this bias may also exist in u-SNLI.