Towards Robustifying NLI Models Against Lexical Dataset Biases
Table 8: The performance of BERT for reducing the WOB via data enhancement/augmentation. The numbers each representing the result after adding 500/20,000/50,000 additional data.
['[BOLD] Train/Test', '[BOLD] MNLI  [BOLD] Acc', '[BOLD] Bal  [BOLD] Acc', '[BOLD] Bal  [BOLD] Acc_hr', '[BOLD] Stress  [BOLD] Acc', '[BOLD] Stress  [BOLD] Acc_hr']
The data augmentation/enhancement results for BERT-base Devlin et al. As for WOB, the gap between Acc and Acc_hr for Bal is much smaller, however, the performance on Stress is very poor. Therefore, we assume that even though BERT achieves a high score on the WOB Bal dataset, BERT is just overfitting the dataset in another different way, i.e., there is still significant WOB bias in BERT. In conclusion, in our experiment, BERT still shows significant CWB and WOB.