What does BERT Learn from Multiple-Choice Reading Comprehension Datasets?
Table 2: Results for un-readable data attacks. Numbers in italics are percentage change relative to the original performance. The most effective attack method on each dataset is in bold.
['[EMPTY]', 'MC160', 'MC500', 'RACE-M', 'RACE-H', 'MCScript', 'MCScript2.0', 'DREAM', 'Average']
The performance of BERT, which is finetuned on the original MCRC dataset, is shown on the top of the table. Note that our attacks do not change the questions and the correct answers, and so the ground-truth of the questions is also not changed. All the methods named with “Shuffle” will add un-readable information to either the passage or the distratcors. Based on the experiment results, we can clearly see that the performance of BERT significantly drops on almost all the datasets, and for some datasets the performance is even lower than random guess. It means that BERT is not able to detect the correct word order or the grammar, and is heavily relying on the keywords matching. For example, BERT drops around 50% on average on all the datasets under the attack of “AddSent2Pas-Shuffle”’. Although the appended data is not readable for human, it is able to fool BERT. We also get a quite large negative correlation coefficient score. It also shows that, in general, changing original sequence length from the datasets is not the main reason why the attacks can fool BERT.