Dynamic Fusion Networks for Machine Reading Comprehension
Table 5: Ablation studies of DFN for Dynamic Fusion (DF) and multi-step reasoning (MR). Results are from 3 ensembles to avoid variance in training. * indicates that the improvement of DFN over the ablation model is significant at the level of p¡0.01.
['[BOLD] Model', 'Acc (dev)%', 'Acc (test)']
To avoid variance in training and provide a fair comparison, 3 ensembles of each model were trained and evaluated on both dev and test sets. This performance boost was contributed by both multi-step reasoning and dynamic fusion. When omitting DF or MR alone, the performance of DFN model dropped by 1.1% and 1.2%, respectively. To validate the effectiveness of the DFN model, we also performed a significance test and compared the full model with each ablation model. The null hypothesis is: the full DFN model has the same performance as the ablation model.