Recurrent Chunking Mechanisms for Long-Text Machine Reading Comprehension
Table 3: F1 Score (%) on documents with different numbers of tokens (max sequence length is 512).
['Dataset # of Doc Tokens', 'CoQA <=200', 'CoQA (200, 300]', 'CoQA (300, 400]', 'CoQA >400', 'QuAC <=300', 'QuAC (300, 450]', 'QuAC (450, 600]', 'QuAC >600']
The gain is more obvious with maximum sequence length (192,256,384), and relatively small under the length of 512. This is perhaps because most document lengths are smaller than 512 in CoQA and QuAC. We observe that the gain is more obvious on longer documents. For documents with more than 400 words in the CoQA dataset, RL chunking with gated recurrence has an improvement of 7.3% over BERT-Large, and RL chunking with LSTM recurrence improves F1 score by 7.5%. As for QuAC, the improvement of gated recurrence with RL chunking is 4.5%, and the improvement of LSTM recurrence is 2.6%.