Dynamic Fusion Networks for Machine Reading Comprehension
Table 3: Accuracy% of DFN compared to baseline methods on RACE test sets. Results of the baseline models came from Lai et al. (2017) and unpublished ElimiNet (∗ indicates ensemble models).
['[BOLD] Model', 'RACE-M', 'RACE-H', 'RACE']
All models were trained with the full RACE dataset, and tested on RACE-M and RACE-H, respectively. As shown in the table, on RACE-M, DFN leads to a 7.8% and 7.3% performance boost over GA and Stanford AR, respectively. On RACE-H, the outperformance is 1.5% and 2.7%. The ensemble models also gained a performance boost of 4-5% comparing to previous methods. We suspect that the lower gain on RACE-H might result from the higher level of difficulty in those questions in RACE-H, as well as ambiguity in the dataset. Human performance drops from 85.1 on RACE-M to 69.4 on RACE-H, which indicates RACE-H is very challenging even for human.