Unlearn Dataset Bias in Natural Language Inference by Fitting the Residual
Table 4: F1 scores of the entailment (E) and non-entailment (¬E) classes on HANS. All models are trained on MNLI and results are shown on three subsets targeting at different biases: lexical overlap (lexical), subsequence overlap (subseq), and constituent overlap (const). Intensity of the Blue and red highlights corresponds to absolute increase and decrease of scores with respect to MLE. DRiFt significantly improves results on challenging ¬E examples without hurting performance on E, whereas Rm improves scores on ¬E at the cost of performance on E.
['method', 'lexical  [ITALIC] E', 'lexical ¬ [ITALIC] E', 'subseq  [ITALIC] E', 'subseq ¬ [ITALIC] E', 'const  [ITALIC] E', 'const ¬ [ITALIC] E']
As expected, models trained by MLE almost always predict entailment (E), and thus performs poorly for the non-entailment class (¬E). DRiFt improves performance on ¬E in all cases with little degradation on E.