Let’s Ask Again: Refine Network for Automatic Question Generation
Table 4: Comparison between Preliminary Decoder and Refinement Decoder in RefNet Model for SQuAD Sentence Level QG.
['Model', '[BOLD] Decoder', '[BOLD] BLEU-4', '[BOLD] QBLEU-4']
The two decoders impact each other through two paths: (i) indirect path, where they share the encoder and the output projection to the vocabulary V, (ii) direct path, via the dual attention network, where the initial draft of the question is attended by the Refinement Decoder. This suggests that generating two variants of the question improves the performance of the first decoder pass as well. This is perhaps due to the additional feedback that the shared encoder and output layer get from the Refinement Decoder. As discussed earlier, Q-Metric measures Answerability using four components, viz., Named Entities, Important Words, Function Words, and Question Type. We observe that the increase in Q-Metric for refined questions is because the RefNet model can correct/add the relevant Named Entities in the question. In particular, we observe that the Named Entity component score in Q-Metric increases from 32.42 for the first draft to 37.81 for the refined draft.