Sentence-Level Content Planning and Style Specificationfor Neural Text Generation
Table 6: Human evaluation on argument generation (Upper) and Wikipedia generation (Bottom). Grammaticality (Gram), correctness (Corr), and content richness (Cont) are rated on Likert scale (1−5). We mark our model with ∗ to indicate statistically significantly better ratings over the variant without style specification (p<0.001, approximate randomization test).
['[EMPTY]', '[BOLD] Argument  [BOLD] Gram.', '[BOLD] Argument  [BOLD] Corr.', '[BOLD] Argument  [BOLD] Cont.', '[BOLD] Wikipedia  [BOLD] Gram.', '[BOLD] Wikipedia  [BOLD] Corr.', '[BOLD] Wikipedia  [BOLD] Cont.']
According to Krippendorff’s α, the raters achieve substantial agreement on grammaticality and correctness, while the agreement on content richness is only moderate due to its subjectivity. However, there is still a gap between system generations and human edited text.