Let’s Ask Again: Refine Network for Automatic Question Generation
Table 6: Impact of Reward-RefNet on fluency and answerability. %preference denotes the percentage of times annotators prefer the generated output from the model for fluency in case of BLEU Reward signal and answerability in case of Answerability Reward signal.
['[BOLD] Model', '[BOLD] BLEU Reward Signal BLEU4', '[BOLD] BLEU Reward Signal %prefer- ence', '[BOLD] Answerability Reward Signal Ans.', '[BOLD] Answerability Reward Signal %prefer ence']
In this section, we analyze the impact of employing different reward signals in Reward-RefNet. We validated these results through human evaluations across 200 samples. Annotators prefer the Reward-RefNet model in 67% of the cases for fluency. Similarly when we use Answerability score as a reward signal, answerability improves for the model and annotators prefer the Reward-RefNet in 70% of the cases for answerability.