Multi-range Reasoning for Machine Comprehension
Table 2: Experimental Results on SearchQA dataset. dunn2017searchqa . Unigram Accuracy and N-gram F1 are reported following kundu2018amanda . All models with † use the same encoder in the answer pointer layer. ∗ are estimates running a replicated model with same batch size (b=256) as our models.
['Model', 'Dev Acc', 'Dev F1', 'Test Acc', 'Test F1', 'Time']
We draw the reader’s attention to the performance of the 300d MRU encoder. We achieve the same accuracy as AMANDA without using any LSTM or GRU encoder. This model runs at 2 min per epoch, making it 4 times more efficient than AMANDA (estimated, with identical batch size). While, AMANDA also uses multi-factor self-attention, along with character enhanced representations, our simple MRU encoder used within a mere baseline bi-attentive framework comes close in performance. Finally, the hybrid combination, MRU-LSTM significantly outperforms AMANDA by 3%.