Multi-hop Reading Comprehension through Question Decomposition and Rescoring
Table 8: The error analyses of human experiment, where the upperbound F1 score of span-based sub-questions with no decomposition scorer is measured.
['Breakdown of 15 failure cases Incorrect groundtruth', 'Breakdown of 15 failure cases 1']
53% of such cases are due to the incorrect groundtruth, partial match with the groundtruth or mistake from humans. 47% are genuine failures in the decomposition. For example, a multi-hop question “Which animal races annually for a national title as part of a post-season NCAA Division I Football Bowl Subdivision college football game?” The question can be decomposed into “Which post-season NCAA Division I Football Bowl Subdivision college football game?” and “Which animal races annually for a national title as part of ANS?” . However in the given set of paragraphs, there are multiple games that can be the answer to the first sub-question. Although only one of them is held with the animal racing, it is impossible to get the correct answer only given the first sub-question. We think that incorporating the original question along with the sub-questions can be one solution to address this problem, which is partially done by a decomposition scorer in DecompRC.