Dynamically Fused Graph Network for Multi-hop Reasoning
Table 2: Ablation study of question answering performances in the development set of HotpotQA in the distractor setting. We use a DFGN with 2-layer fusion blocks as the origin model. The upper part is the model ablation results and the lower part is the dataset ablation results.
['Setting', 'EM', 'F1']
To evaluate the performance of different components in our DFGN, we perform ablation study on both model components and dataset segments. Here we follow the experiment setting in \newciteyang2018hotpotqa to perform the dataset ablation study, where we only use golden paragraphs or supporting facts as the input context. From the table we can see that each of our model components can provide from 1% to 2% relative gain over the QA performance. Particularly, using a 1-layer fusion block leads to an obvious performance loss, which implies the significance of performing multi-hop reasoning in HotpotQA. Besides, the dataset ablation results show that our model is not very sensitive to the noisy paragraphs comparing with the baseline model which can achieve a more than 5% performance gain in the “gold paragraphs only” and “supporting facts only” settings. Yang et al.