A Knowledge-Enhanced Pretraining Model for Commonsense Story Generation
Table 6: Final prediction F1 score of our model on the auxiliary story classification task in terms of the four types of story sets respectively, and the proportional distribution of the predicted story types of the generated stories by different models.
['[BOLD] Story types', '[BOLD] D1', '[BOLD] D2', '[BOLD] D3', '[BOLD] D4']
Both metrics are computed on 1,000 samples from the test set. We can observe that it is relatively easier to detect fake stories with repeated plots (D4) than those with disordered logic (D2) and unrelated topics (D3). When using the auxiliary story classifier to classify the generated stories, pretrained GPT-2 is considered to generate more fake stories, with only 15.83% stories of type D1, which agrees with the previous automatic evaluation especially in terms of repetition. Besides, our model performs better than baselines, indicating that the external knowledge and the auxiliary task can encourage our model to generate more reasonable stories.