Simple and Effective Multi-Paragraph Reading Comprehension
Table 2: Results on TriviaQA web using our pipelined method. We significantly improve upon the baseline by combining the preprocessing procedures, TF-IDF paragraph selection, the sum objective, and our model design.
['Model', 'EM', 'F1']
First, we do an ablation study on TriviaQA web to show the effects of our proposed methods for our pipeline model. We start with an implementation of the baseline from Joshi et al. Their system selects paragraphs by taking the first 400 tokens of each document, uses BiDAF Seo et al. ’s cross entropy loss function during training. Paragraphs of size 800 are used at test time. significantly, likely because we are not subsampling the data. We find both TF-IDF ranking and the sum objective to be effective; even without changing the model we achieve state-of-the-art results. Using our refined model increases the gain by another 4 points.