Semantics of the Unwritten
Table 3: Test results on ChineseEssay dataset.
['[BOLD] ParaType', '[BOLD] FT', '[BOLD] T PPL', '[BOLD] T PPL(-)', '[BOLD] BLEU1', '[BOLD] BLEU2', '[BOLD] DIST1', '[BOLD] DIST2', '[BOLD] EOS%', '[BOLD] EOS PPL']
score: Chinese-GPT2 without any newly-added token performs best on this metric. This is reasonable considering the model size and the data size: introducing newly initialized tokens(EOP) would hinder the perplexity of this model. Besides, we can find that the improvements on T PPL mainly comes from the reduction of EOS PPL, plunging from 22.15 to 2.74. And the EOS% rising from 76.41 to 93.07, indicating that more generated essays end with the EOS after learning to end paragraphs. Finally, different from DIST scores of the English stories, EOS DIY got the highest DIST1 and DIST2 scores among all models, while the ground truth essays get 41.2 DIST1 and 82.65 DIST2.