RC-QED: Evaluating Natural Language Derivationsin Multi-Hop Reading Comprehension
Table 5: Performance breakdown of the PRKGC+NS model. Derivation Precision denotes ROUGE-L F1 of generated NLDs.
['# gold NLD steps', 'Answer Prec.', 'Derivation Prec.']
As shown in Table 5, as the required derivation step increases, the PRKGC+NS model suffers from predicting answer entities and generating correct NLDs. [CONTINUE] This indicates that the challenge of RC-QEDE is in how to extract relevant information from supporting documents and synthesize these multiple facts to derive an answer.