A Simple but Effective Method to Incorporate Multi-turn Context with BERT for Conversational Machine Comprehension
Table 4: Ablation study on the development set of QuAC and CoQA (F1 score)
['[EMPTY]', 'CoQA', 'QuAC']
S5SS3SSS0Px3 Which is more important, the question history or the answer history? We can see from the results that the model performance decreased significantly when we removed the gold answer history on QuAC. In dataset collection, CoQA allows the asker to see the evidence paragraph. On the other hand, the asker in QuAC cannot see the evidence paragraph. As a result, questions in QuAC are far from the phrases in the passage and are less effective in improving the model performance. For CoQA, the model could substitute the question history for the gold answer history. The model performance did not decrease significantly when we remove the answer history. This result also suggests that the model can substitute the question history for the gold answer history in CoQA. We think the CMC setting where the history of questions posed by an asker that does not see the evidence paragraph is given and the gold answer is not given for input is a more realistic and important setting.