Robust Question Answering Through Sub-part Alignment
Table 2: Performance of our systems compared to the literature on both addSent and addOneSent. Here, overall denotes the performance on the full adversarial set, adv denotes the performance on the adversarial samples alone. Δ represents the performance gap between the normal SQuAD and the overall performance on adversarial set.
['[EMPTY]', 'Normal', 'addSent overall', 'addSent adv', 'addSent Δ', 'addOneSent overall', 'addOneSent adv', 'addOneSent Δ']
We note that the overall performance of our model on the two adversarial set is lower compared with BERT, while the performance on adversarial samples alone is higher. That is because we make the task harder – we trade some in-distribution performance to improve the model’s robustness, controllability and explainability. Also, we see that our model achieves the smallest gap on addSent and a comparable gap on addOneSent, which demonstrates that the constrained alignment we proposed is a strong and effective way to enhance the robustness of the model compared to some of previous methods like adversarial training Yang et al.