Reinforcement Learning Based Graph-to-Sequence Model for Natural Question Generation
Table 3: Ablation study on the SQuAD split-2 test set.
['Methods G2S [ITALIC] dyn+BERT+RL', 'BLEU-4 18.06', 'Methods G2S [ITALIC] dyn', 'BLEU-4 16.81']
(static vs dynamic) on the SQuAD split-2 test set. It confirms our finding that syntax-based static graph construction (G2Ssta+BERT+RL) performs better than semantics-aware dynamic graph construction (G2Sdyn+BERT+RL) in almost every setting. However, it may be too early to conclude which one is the method of choice for QG. On the one hand, an advantage of static graph construction is that useful domain knowledge can be hard-coded into the graph, which can greatly benefit the downstream task. However, it might suffer if there is a lack of prior knowledge for a specific domain knowledge. On the other hand, dynamic graph construction does not need any prior knowledge about the hidden structure of text, and only relies on the attention matrix to capture these structured information, which provides an easy way to achieve a decent performance. One interesting direction is to explore effective ways of combining both static and dynamic graphs.