Cognitive Graph for Multi-Hop Reading Comprehension at Scale
Table 1: Results on HotpotQA (fullwiki setting). The test set is not public. The maintainer of HotpotQA only offers EM and F1 for every submission. N/A means the model cannot find supporting facts.
['[EMPTY]', 'Model', 'Ans EM', 'Ans  [ITALIC] F1', 'Ans Prec', 'Ans Recall', 'Sup EM', 'Sup  [ITALIC] F1', 'Sup Prec', 'Sup Recall', 'Joint EM', 'Joint  [ITALIC] F1', 'Joint Prec', 'Joint Recall']
The leap of performance mainly results from the superiority of the CogQA framework over traditional retrieval-extraction methods. Since paragraphs that are multi-hop away may share few common words literally or even little semantic relation with the question, retrieval-extraction framework fails to find the paragraphs that become related only after the reasoning clues connected to them are found. Our framework, however, gradually discovers relevant entities following clues.