Adversarial Domain Adaptation for Machine Reading Comprehension
Table 4: Results of using ELMo and BERT. Setting: adaptation from SQuAD to NewsQA.
['Method', 'EM/F1']
To verify that our approach is compatible with large-scale pre-trained language models, we evaluate our model with ELMo Peters et al. We use the original design of finetuning BERT for the MRC task in Devlin et al. We observe that using ELMo and BERT improves both AdaMRC and the baseline model. However, the improvement over ELMo and BERT is relatively smaller than SAN. We believe this is because pre-trained language model provides additional domain-invariant information learned from external data, and therefore limits the improvement of domain-invariant feature learning in our model. However, it is worth noting that combining AdaMRC with BERT achieves the best performance, which validates that AdaMRC is compatible with data augmentation from external sources.