Ask to Learn: A Study on Curiosity-driven Question Generation
Table 2: Results obtained on QuAC-derived data.
['[EMPTY]', '[BOLD] human', '[BOLD] base_beam1', '[BOLD] base_beam3', '[BOLD] base_beam5', '[BOLD] RL_beam1', '[BOLD] RL_beam3', '[BOLD] RL_beam5']
We use a beam k, and compute the results for k=[1,3,5]. In addition the generated questions with a beam k=5, we also computed the results for k=1 and k=3. While one would expect to see for all the metrics a slight improvement, with increasing beam size, we observe a strong divergence among the results: increasing values for k correspond to a significant improvements in terms of BLEU-4 and notable drops for BLEU-1. A similar phenomena was observed by Ott et al. In our case, one of most frequent generated question is Are there any other interesting aspects about this article ?. Indeed, the frequency of this question in our training set amounts to 4.18% of the questions. On the test set we see that roughly 80% of the generated questions start with the token “are” . Generating this sequence is not very likely with a greedy search (k=1): at any time step during the generation, if any other token has a higher probability, this question will be dismissed. On the other hand, with a higher beam, it is likely to be kept and eventually result as the most probable sequence, among the different remaining beams at the end of the inference.