End-to-End Bias Mitigation by Modelling Biases in Corpora
Table 5: Results on MNLI mismatched dev set, MNLI mismatched hard set, and HANS when training independently to debias against either hypothesis artifacts (♣) or syntactic biases (♥), compared with jointly training to debias against both bias types. Δ: differences with baseline CE loss.
['[BOLD] Loss CE', '[BOLD] MNLI 84.53', '[BOLD] Hard 77.55', 'Δ', '[BOLD] HANS 61.88±1.9', 'Δ']
Models trained to be robust to hypothesis biases (♣) do not generalize to HANS. On the other hand, models trained to be robust on HANS (♥) use a powerful bias-only model resulting in a slight improvement on MNLI mismatched hard dev set. We expect a slight degradation when debiasing for both biases since models need to select samples accommodating both debiasing needs. The jointly debiased models successfully obtain improvements on both datasets, which are close to the improvements on each dataset by the individually debiased models.