RC-QED: Evaluating Natural Language Derivationsin Multi-Hop Reading Comprehension
Table 4: Performance of RC-QEDE of our baseline models (see Section 2.1 for further details of each evaluation metrics). “NS” indicates the use of annotated NLDs as supervision (i.e. using Ld during training).
['Model', 'Answerability Macro P/R/F', '# Answerable', 'Answer Prec.', 'Derivation Prec. RG-L (P/R/F)', 'Derivation Prec. BL-4']
As shown in Table 4, the PRKGC models learned to reason over more than simple shortest paths. [CONTINUE] Yet, the PRKGC model do not give considerably good results, which indicates the non-triviality of RC-QEDE. [CONTINUE] Although the PRKGC model do not receive supervision about human-generated NLDs, paths with the maximum score match human-generated NLDs to some extent. [CONTINUE] Supervising path attentions (the PRKGC+NS model) is indeed effective for improving the human interpretability of generated NLDs. [CONTINUE] It also improves the generalization ability of question answering.