Read + Verify: Machine Reading Comprehension with Unanswerable Questions
Table 2: Comparison of readers with different auxiliary losses.
['Configuration', 'HasAns EM', 'HasAns F1', 'All EM', 'All F1', 'NoAns ACC']
Next, we do an ablation study on the SQuAD 2.0 development set to show the effects of our proposed methods for each individual component. Removing the independent span loss (indep-I) results in a performance drop for all answerable questions (HasAns), indicating that this loss helps the model in better identifying the answer boundary. Ablating independent no-answer loss (indep-II), on the other hand, causes little influence on HasAns, but leads to a severe decline on no-answer accuracy (NoAns ACC). This suggests that a confliction between answer extraction and no-answer detection indeed happens. Finally, deleting both of two losses causes a degradation of more than 1.5 points on the overall performance in terms of F1, with or without ELMo embeddings.