Multi-range Reasoning for Machine Comprehension
Table 3: Experimental Results on the NarrativeQA reading comprehension challenge kovcisky2017narrativeqa using summaries. † are baselines reported by kovcisky2017narrativeqa .
['Model', 'Bleu-1', 'Bleu-4', 'Meteor', 'Rouge-L', 'Time']
When compared with a BiLSTM of equal output dimensions (150d), we find that our MRU model performs competitively, with less than 1% deprovement across all metrics. However, the time cost required is significantly reduced. The performance of our model is significantly better than 300d LSTM model while also being significantly faster. Here, we note that Sim. MRU does not produce reasonable results at all, which seems to be in similar vein to results on SearchQA, i.e., a recursive cell that processes word-by-word is mandatory for span prediction. However, our results show that it is not necessary to construct gates in a word-by-word fashion. Finally, the MRU-LSTM significantly outperforms all models, including BiDAF on this dataset. Performance improvement over the vanilla BiLSTM model ranges from 1%−3% across all metrics, suggesting that MRU encoders are also effective as a complementary neural building block.