CAiRE-COVID: A Question Answering and Multi-Document Summarization System for COVID-19 Research
Table A1: Results of the MRQA model on unseen datasets Su et al. (2019). TQA, RE and CWQ are the abbreviations for TextbookQA, RelationExtraction and ComplexWebQuestions respectively.
['Datasets', '[BOLD] MRQA model  [BOLD] EM', '[BOLD] MRQA model  [BOLD] F1', '[BOLD] Baseline  [BOLD] EM', '[BOLD] Baseline  [BOLD] F1']
MRQA model Su et al. To equip the model with better generalization ability to unseen data, the MRQA model is trained in multi-task learning scheme on six datasets, which are SQuAD Rajpurkar et al. SearchQA Dunn et al. HotpotQA Yang et al. and NaturalQuestions Kwiatkowski et al. The training sets vary from each other in terms of data source, context lengths, whether multi-hop reasoning is needed, strategies for data augmentation. To evaluate the generalization ability, the authors utilized BERT-large model Devlin et al. The models are evaluated on twelve unseen datasets, such as DROP Dua et al. et al.