End-to-End Bias Mitigation by Modelling Biases in Corpora
Table 6: Accuracy results of models with BERT transferring to new target datasets. All models are trained on SNLI and tested on the target datasets. Δ are absolute differences between our methods and the CE loss baseline.
['[BOLD] Data SICK', '[BOLD] CE 57.05', '[BOLD] DFL 57.91', 'Δ +0.9', '[BOLD] PoE 57.28', 'Δ +0.2']
As shown in prior work (belinkov-etal-2019-dont), the MNLI datasets have very similar biases to SNLI, which the models are trained on, so we do not expect any improvement in the relative performance of our models and the baseline for MNLI and MNLI-M. On all the remaining datasets, our proposed models perform better than the baseline, showing a substantial improvement in generalization by using our debasing techniques.