RikiNet: Reading Wikipedia Pages for Natural Question Answering
Table 1: Performance comparisons on the dev set and the blind test set of the NQ dataset. We report the evaluation results of the precision (P), the recall (R), and the F1 score for both long-answer (LA) and short-answer (SA) tasks. We use background color to highlight the column of F1 results. † refers to the works that only provide the F1 results on the dev set in their paper. ‡ refers to our implementations where we only report the results on the dev set, due to the NQ leaderboard submission rules (each participant is only allowed to submit once per week).
['[EMPTY]', 'LA Dev P', 'LA Dev R', 'LA Dev F1', 'LA Test P', 'LA Test R', 'LA Test F1', 'SA Dev P', 'SA Dev R', 'SA Dev F1', 'SA Test P', 'SA Test R', 'SA Test F1']
We present a comparison between previously published works on the NQ task and our RikiNet. We report the results of the precision (P), the recall (R), and the F1 score for the long-answer (LA) and short-answer (SA) tasks on both test set and dev set in Tab. The first two lines of Tab. The third to sixth lines show the results of the previous state-of-the-art models. These models all employ the BERTlarge model and perform better than that two baselines. Our RikiNet-BERTlarge also employs the BERTlarge model, and its single model has achieved a significant improvement over the previously published best model on the test set (LA from 66.8 F1 to 74.3 F1, and SA from 53.9 F1 to 57.9 F1). We also provide a BERTjoint alberti2019bert + RoBERTa large liu2019roberta baseline on NQ, which only replaces the BERTlarge in BERTjoint method with RoBERTa large. To be expected, the BERTjoint + RoBERTa large performs better than original BERTjoint. Furthermore, our single model of RikiNet-RoBERTa large which employs RoBERTa large model also achieves better performance on both LA and SA, significantly outperforming BERTjoint + RoBERTa large. These results demonstrate the effectiveness of our RikiNet.