Recurrent Chunking Mechanisms for Long-Text Machine Reading Comprehension
Table 5: F1 score (%) of the BERT-Large model with different training/prediction stride sizes on the CoQA and QuAC datasets.
['Dataset BERT-Large bert2018', 'CoQA Prediction Stride Size', 'CoQA Prediction Stride Size', 'CoQA Prediction Stride Size', 'CoQA Prediction Stride Size', 'QuAC Prediction Stride Size', 'QuAC Prediction Stride Size', 'QuAC Prediction Stride Size', 'QuAC Prediction Stride Size']
An interesting observation is that smaller stride size in prediction does not always improve the performance, sometimes even hurts as can be seen on the QuAC dataset. It suggests that BERT-Large performs badly on selecting good answers from multiple chunks. Smaller stride size in model training also leads to worse performance. A possible explanation is that smaller stride size would cause the significant distortion of training data distribution, since the longer question-document pairs produces more training samples than short ones.