What does BERT Learn from Multiple-Choice Reading Comprehension Datasets?
Table 3: Results for shuffled and partial data training.
['[EMPTY]', 'MC160', 'MC500', 'RACE-M', 'RACE-H', 'MCScript', 'MCScript2.0', 'DREAM']
From the experiments, we can clearly see that BERT trained on all these settings are much better than random guess and “Longest Baseline” - which always selects the longest option as the prediction. For example, BERT trained under the setting of “P-Shuffle”, which randomly shuffles all the passage words in the training set, even gets very close performance to the original full training setting, especially on the MCScript and MCScript2.0 datasets. If we further compare “P-Shuffle” and “P-Remove”, we can find that although the shuffled passage does not follow the correct grammar and is not interpretable any more, it is still better than removing the whole passage. Similar performance can also be found by comparing “Q-Shuffle” and “Q-Remove”, “PQ-Shuffle” and “PQ-Remove”. In this way, BERT can still learn from the shuffled input, which suggests that it is insensitive to the word order or the syntactic information. For example, according to the performance of “Q-Remove” and “P-Remove”, without even reading the question or the passage, BERT can already achieve much better performance than random guess, or sometimes even close to the full training setting (for instance, BERT achieves 76.8% accuracy on MCScript and 73.6% accuracy on MCScript2.0 test set when trained without the passages.) This suggests that BERT may be exploiting dataset artifacts and statistical cues to achieve high performance. In this way, to test the true ability of neural models to comprehend the passages, questions, options and perform reasoning, we need to be more careful when constructing the datasets.