Template-Based Question Generation from Retrieved Sentences for Improved Unsupervised Question Answering
Table 1: Effect of original vs retrieved sentences for generic cloze-style question generation.
['Training procedure', 'EM', 'F1']
Effect of retrieved sentences: We test the effect of retrieved vs original sentences as input to question generation when using generic cloze questions. The retrieval process may return sentences which do not match the original context. On a random sample, 15/18 retrieved sentences were judged as entirely relevant to the original sentence. This retrieval is already quite good, as we use a high quality ElasticSearch retrieval and use the original context sentence as the query, not just the answer word. While we do not explicitly ensure that the retrieved sentence has the same meaning, we find that the search results with entity matching gives largely semantically matching sentences. Additionally, we believe the sentences which have loosely related meaning may act as a regularization factor which prevent the downstream QA model from learning only string matching patterns. Along these lines, Lewis et al. We believe that loosely related context sentences can act as a more intuitive noise function, and investigating the role of the semantic match of the retrieved sentences is an important direction for future work. For the sections which follow, we only show results of retrieved sentences, as the trend of improved performance held across all experiments.