Attention-Guided Answer Distillation for Machine Reading Comprehension
Table 2: Comparison of different approaches on two adversarial SQuAD datasets: min2018efficient1. (E: ensemble model)
['Model', 'AddSent EM', 'AddSent F1', 'AddOneSent EM', 'AddOneSent F1']
As we can see, the improvement on adversarial data is much higher than the one on original SQuAD dataset: the student network successfully surpasses the teacher. The significant improvement comes from the fact that there exists much more confusing answers in adversarial datasets, and hence the baseline is more likely to be over-confident on distractors. Our approach, however, explicitly decreases distractor’s confidence, thus yielding more robust predictions against adversarial examples.