Toward Subgraph Guided Knowledge Graph Question Generation with Graph Neural Networks
Table 2: Evaluation results on WQ and PQ.
['Method', 'BLEU-4', 'WQ METEOR', 'ROUGE-L', 'BLEU-4', 'PQ METEOR', 'ROUGE-L']
As we can see, our models outperform all baseline methods by a large margin on both benchmarks. Besides, we can clearly see the advantages of GNN-based encoders for modeling KG subgraphs, by comparing our model with RNN-based (i.e., L2A) and Transformer-based (i.e., Transformer, MHQG+AE) baselines. Compared to our Graph2Seq model, both RNN-based and Transformer-based baselines ignore the explicit graph structure of a KG subgraph, which leads to degraded performance. Interestingly, the Transformer baseline performs reasonably well on PQ, but dramatically fails on WQ. We speculate this is because PsQ is more friendly to sequential models such as Transformer as the KG subgraph in PQ is more like path-structure while the one in WQ is more like tree-structure. We also compare two variants (i.e., G2S vs. G2Sedge) of our model for handling multi-relational graphs. The proposed BiGGNNedge model can directly handle multi-relational graphs without modifying the input graph. However, it performs slightly worse than the Levi graph solution. We suspect that it might help improve the modeling power of BiGGNNedge by updating edge embeddings in the message passing process and attending to edges in the attention mechanism. Currently, these two features are missing in BiGGNNedge. We leave these as future work.