An Empirical Analysis of Multiple-Turn Reasoning Strategies in Reading Comprehension Tasks
Table 6: Effect of multiple passages per query in MS MARCO.
['[EMPTY]', 'BLEU/ROUGE-L']
Recall that the RC model needs to read approximately 10 passages to answer each query, and on average only one or two passage contain answer spans. ReasoNet++ extracts answer spans from each passage independently, then combines with an IR model to output the final answer. If we assume oracle ranking from the IR model, we can achieve 62 BLEU / 63 ROUGE, suggesting that better passage ranking models (e.g. via joint training with RC models) is fruitful as future work.