MRQA 2019 Shared Task: Evaluating Generalization in Reading Comprehension
Table 5: Pretrained language models used in the shared task submissions. *-L and *-B denote large and base versions of the models.
['[BOLD] Model', '[BOLD] Base', '[BOLD] Eval F1']
The top three performing systems all use XLNet instead of BERT-Large—this isolated change in pre-trained language model alone yields a significant gain in overall in- and out-of-domain performance. Li et al. Su et al.