Easy-to-Hard: Leveraging Simple Questions for Complex Question Generation
Table 4: CQG with sub-questions from WebQ. Superscript “pseudo” indicates training with randomly sampled pseudo sub-questions rather than real ones. * denotes significantly different from CoG2Q6pt+data in one-tailed t-test (p<0.05).
['[EMPTY]', '[BOLD] BLEU-4', '{ROUGE}{L}', '[BOLD] METEOR']
Using sub-questions. In particular, we make the following observations: (1) The straightforward data augmentation strategy (+ data) improves the performance of CoG2Q by incorporating the sub-questions as extra training samples. Note that for fair comparison, we have used all the sub-questions (corresponding to not only training, but also validation and testing complex questions) as extra training samples. (2) The SubQ baseline simply compares the closeness of a sub-question to a complex question. This shows that CoGSub2G needs to do more than just copy every word from the sub-questions. (3) It is verified by the results that data augmentation is outperformed by CoGSub2Q, which can explicitly encode and copy from sub-questions. (4) Very interestingly, we also discover that the sub-question correspondence information is essential to train the CoGSub2Q model successfully. Specifically, we experiment with CoGSub2Q in two different scenarios, one trained with randomly sampled pseudo sub-questions and the other with actual sub-questions. The reason for this is similar to why data augmentation is not optimal: the sub-question to complex question mapping is less clearly established when the model learns from pseudo sub-questions instead of actual ones. Conversely, learning with sub-questions better exploits the instance-level connections, and it is easier for the model to learn with the actual sub-questions that often contain more words appearing in the target complex questions.