Asking Questions the Human Way: Scalable Question-Answer Generation from Text Corpus
Table 3. Evaluating the question answering performance with different training datasets.
['[BOLD] Experiments', '[BOLD] CS2S-VR-ACS  [BOLD] EM', '[BOLD] CS2S-VR-ACS  [BOLD] F1', '[BOLD] GPT2-ACS  [BOLD] EM', '[BOLD] GPT2-ACS  [BOLD] F1']
Our implementation gives 86.72% EM and 92.97% F1 when trained on the original SQuAD dataset. In comparison, the model trained by our generated dataset gives 74.47% and 85.64% F1. When we combine the generated dataset with the SQuAD training set to train the QA model, the performance is not further improved. The results are reasonable. First, the generated dataset contains noises which will influence the performance. Second, simply increasing the size of training dataset will not always help with improving the performance. If most of the generated training samples are already answerable by the model trained over the original SQuAD dataset, they are not very helpful to further enhance the generalization ability of the model.