Multi-hop Reading Comprehension through Question Decomposition and Rescoring
Table 7: Left: ablations in sub-questions. F1 score on a sample of 50 bridging questions from the dev set of HotpotQA, Pointerc is our span-based model trained with 200 or 400 annotations. Right: ablations in decomposition decision method. F1 score on the dev set of HotpotQA with ablating decomposition decision method. Oracle indicates that the ground truth reasoning type is selected.
['Sub-questions', 'F1']
There is little difference in model performance between span-based and sub-questions written by human. This indicates that our span-based sub-questions are as effective as free-form sub-questions. In addition, Pointer3 trained on 200 or 400 examples obtains close to human performance. We think that identifying spans often rely on syntactic information of the question, which BERT has likely learned from language modeling. We use the model trained on 200 examples for DecompRC to demonstrate sample-efficiency, and expect performance improvement with more annotations.