Semantics of the Unwritten
Table 2: Test results of different models with/without finetuning(FT) on WritingPrompts dataset.
['[BOLD] ParaType', '[BOLD] FT', '[BOLD] W PPL', '[BOLD] W PPL(-)', '[BOLD] T PPL', '[BOLD] T PPL(-)', '[BOLD] EOS PPL', '[BOLD] BLEU1', '[BOLD] BLEU2', '[BOLD] DIST1', '[BOLD] DIST2']
Concatenating all paragraphs with “\n” as the separator between paragraphs; SEP DIY: Concatenating all paragraphs with a new token “[SEP]” as the separator between paragraphs; EOP \n: Appending “\n” to the end of each paragraph; EOP DIY: Appending a new token “[EOP]” to the end of each paragraph. Compared with the model finetuned without paragraph information, finetuning with “\n” as SEP can further reduce the perplexity of general words/tokens, and all the models with EOP/SEP achieve better BLEU scores. It should be noticed that the BLEU score is affected by the length of the text. We further truncate all results with the length of the ground truth story to calculate the truncated BLEU score. The average length of None, SEP \n, EOP DIY is 392.7, 367.4, and 379.61, and the truncated BLEU1 is 36.6, 37.6, and 38.6, the truncated BLEU2 is 5.3, 5.0, and 8.1. Thus the overall trending is consistent.