Robust Machine Comprehension Models via Adversarial Training
Table 5: F1 Performance of the BSAE model trained on datasets with different distractor placement strategies.
['Training', 'AddSent', 'AddSentPrepend', 'Average']
This method of calculating the distribution of Psa allows us to predict the model’s performance when trained on datasets where the distractors are inserted at specific locations. To test this hypothesis, we created two datasets: InsFront-3 and InsFront-6 where the distractors were inserted as the 3rd and 6th sentence from the beginning and measure the model’s performance when trained on these two datasets. The distributions of Psa for these two datasets are shown in Fig.