An Iterative Polishing Framework based on Quality Aware Masked Language Model for Chinese Poetry Generation
Table 1: The BLEU score results on different generated poem line with same extracted keyword or previous poem lines. BL-1 and BL-2 are the BLEU scores on unigrams and bigrams.
['[BOLD] Model', '[BOLD] key → 1  [BOLD] BL-1', '[BOLD] key → 1  [BOLD] BL-2', '[BOLD] 1 → 2  [BOLD] BL-1', '[BOLD] 1 → 2  [BOLD] BL-2', '[BOLD] 1&2 → 3  [BOLD] BL-1', '[BOLD] 1&2 → 3  [BOLD] BL-2', '[BOLD] 3 → 4  [BOLD] BL-1', '[BOLD] 3 → 4  [BOLD] BL-2', '[BOLD] Average  [BOLD] BL-1', '[BOLD] Average  [BOLD] BL-2']
The compared models are shown in the first column of the table, where the suffix -P indicates that the poems generated by previous models have been polished by QA-MLM. The keywords and poem lines are extracted from test dataset with one thousand poems. In general, the BLEU scores of CVAE-D are higher than AS2S but lower than B&T, which partially reflects the generation performance of these models. The polishing process improves the BLEU scores for AS2S model while has no obvious improvement or even a bit hurt of BLEU scores for CVAE-D and B&T. This is probably due to the creativity and diversity properties in poem generation. The quality of generated poem is not proportional to the BLEU score when the BLEU score is comparative high. The BLEU score should be referred conservative during the quality evaluation for poetry generation.