Robust Question Answering Through Sub-part Alignment
Table 3: The performance of our model on SQuAD-Adversarial-Triggers. Compared with BERT, our model sees smaller performance drops on all triggers.
['type', 'Sub-part Alignment Normal', 'Sub-part Alignment Trigger', 'BERT Normal', 'BERT Trigger']
We see that every trigger results in a bigger performance drop on BERT QA than our model. Our model is much more stable, especially on who and when question types, in which case the performance only drops by around 2%. Several factors may contribute to the stability: (1) The triggers are ungrammatical and their arguments often contain seemingly random words, which are likely to get lower alignment scores. (2) Because our model is structured and trained in a different fashion than BERT, adversarial attacks designed for span-based question answering model may not fool our model as effectively.