Sparse Text Generation
Table 4: Human evaluation of story completion. All scores marked in bold at each column outperform the others with statistical significance, according to the Wilcoxon’s test with p-value <0.05.
['[EMPTY]', 'Fluency', 'Coherence', 'Engagement']
We performed human evaluation of greedy decoding, top-k, nucleus, and entmax sampling on completion of stories from the WritingPrompts datasets. To perform the human evaluation, we divided 100 stories into 2 sets of 50 stories. For each set of stories, 2 judges evaluated each story in 3 metrics: fluency (whether the text is syntactically and semantically correct), coherence (whether the story continuation is related to the context and is consistent), and engagement (whether the annotator felt interested in the story).