Reinforced Mnemonic Reader for Machine Reading Comprehension
Table 3: Performance comparison on two adversarial SQuAD datasets. Wang & JiangWang17a1, Seo et al.Seo172, Liu et al.Liu17b3, Shen et al.Shen164 and Huang et al.Huang175. ∗ indicates ensemble models.
['[ITALIC] Model', '[BOLD] AddSent  [BOLD] EM', '[BOLD] AddSent  [BOLD] F1', '[BOLD] AddOneSent  [BOLD] EM', '[BOLD] AddOneSent  [BOLD] F1']
All models are trained on the original train set of SQuAD, and are tested on the two datasets. As we can see, R.M-Reader comfortably outperforms all previous models by more than 6% in both EM and F1 scores, indicating that our model is more robust against adversarial attacks.