A Self-Training Method for Machine Reading Comprehension with Soft Evidence Extraction
Table 6: Answer prediction accuracy (Ans. Acc) and evidence extraction accuracy (Evi. Acc) on the development set of CoQA.
['Model/Metric', 'Ans. Acc', 'Evi. Acc']
To evaluate the improvement of STM over stronger pretrained models, we employed RoBERTa-large STM significantly improved the evidence extraction (Evi. Acc) of the base model. However, the improvement on answer prediction (Ans. Acc) is marginal. One reason is that RoBERTa-HA achieved so high performance that there was limited room to improve. Another possible explanation is that evidence information is not important for such stronger models to generate answers. In other words, they may be more adept at exploiting data bias to make answer prediction. In comparison, weaker pretrained models, such as BERT-base, can benefit from evidence information due to their weaker ability to exploit data bias.