Natural Question Generation with Reinforcement Learning Based Graph-to-Sequence Model
Table 2: Ablation study on the SQuAD split-2 test set.
['Methods G2S [ITALIC] dyn+BERT+RL', 'BLEU-4 18.06', 'Methods G2S [ITALIC] dyn', 'BLEU-4 16.81']
By turning off DAN, the BLEU-4 score of G2Ssta (similarly for G2Sdyn) dramatically drops from 16.96% to 12.62%, which shows the effectiveness of DAN. We can see the advantages of Graph2Seq learning over Seq2Seq learning by comparing the performance between G2Ssta and Seq2Seq. Fine-tuning the model using REINFORCE can further improve the model performance, which shows the benefits of directly optimizing evaluation metrics. We also find that BERT has a considerable impact on the performance. Lastly, we find that static graph construction slightly outperforms dynamic graph construction.