Dual Multi-head Co-attention for Multi-choice Reading Comprehension
Table 10: Results using BERT as encoder on DREAM dataset. Results of BERTbase are our re-running.
['model', 'dev', 'test', 'avg']
Though the proposed DUMA is supposed to enhance state-of-the-art pre-trained LM like ALBERT, we argue that it is generally effective for less advanced models. Thus we simply replace the adopted ALBERT by its early variant BERT to examine the effectiveness of DUMA. We see that our model can be easily transferred to other pre-trained LMs, and it can be seemed as an efficient module for modeling relationships among passage, question and answer for MRC.