NumNet: Machine Reading Comprehension with Numerical Reasoning
Table 3: Performance with different GNN structure. “Comparison”, “Number” and “ALL” denote the comparing question subset, the number-type answer subset, and the entire development set, respectively.
['Method', 'Comparison EM', 'Comparison F1', 'Number EM', 'Number F1', 'ALL EM', 'ALL F1']
In this part, we investigate the effect of different GNN structures on the DROP development set. If we replace the proposed numerically-aware graph (Sec. Moreover, “- question num” denotes the numbers in the question is not included in the graph, and “- ≤ type edge” and “- > type edge” denote edges of ≤ and > types are not adopted respectively. It indicates that considering the comparing information over numbers could effectively help the numerical reasoning for comparing questions. Moreover, we find that the numbers in the question are often related to the numerical reasoning for answering the question, thus considering numbers in questions in NumGNN achieves better performance. And the results also justify that encoding “greater relation” and “lower or equal relation” simultaneously in the graph also benefits our model.