Coarse-to-Fine Question Answering for Long Documents
Table 3: Answer prediction accuracy on the test set. K is the number of sentences in the document summary.
['Dataset', 'Learning', 'Accuracy']
We use Bow encoder for sentence selection as it is the fastest. The proposed hierarchical models match or exceed the performance of Base, while reducing the number of RNN steps significantly, from 300 to 35 (or 70 for K=2), and allowing access to later parts of the document. While throughput at training time can be improved by increasing the batch size, at test time real-life QA systems use batch size 1, where Reinforce obtains a 3.5x-6.7x speedup (for K=2 or K=1). In all settings, Reinforce was at least three times faster than the Base model.