An Iterative Polishing Framework based on Quality Aware Masked Language Model for Chinese Poetry Generation
Table 2: The evaluation results. Sim12 refers to the similarity between first poem line and second poem line; Sim34 refers to the similarity between the third poem line and the fourth poem line; Sim2L refers to the similarity between first two poem lines and the last two poem lines; TA. and RA. are the tone level predicted accuracy and the rhythm predicted accuracy respectively; Con., Flu., Mea., and Poe. represent the Consistency, Fluency, Meaningfulness and Poeticness respectively.
['[BOLD] Model', '[BOLD] Automatic Evaluation  [BOLD] Sim12', '[BOLD] Automatic Evaluation  [BOLD] Sim34', '[BOLD] Automatic Evaluation  [BOLD] Sim2L', '[BOLD] Automatic Evaluation  [BOLD] TA.', '[BOLD] Automatic Evaluation  [BOLD] RA.', '[BOLD] Human Evaluation  [BOLD] Con.', '[BOLD] Human Evaluation  [BOLD] Flu.', '[BOLD] Human Evaluation  [BOLD] Mea.', '[BOLD] Human Evaluation  [BOLD] Poe.']
These evaluation results are also based on one thousand poems from test dataset. The keywords are extracted from these poems and utilized to generate poem drafts by AS2S, CVAE-D and B&T. All the poem drafts generated by this three encoder-decoder structure models have been further polished by our proposed QA-MLM model. We can find that the polish procedure improves the scores of Similarity criteria on all the compared encoder-decoder structure models, which demonstrates the effectiveness of our proposed QA-MLM for the improvement of sematic coherence and theme consistency. The tone level and rhythm for AS2S and CVAE-D models seem randomly. By contrast, the accuracy of tone level and rhythm for B&T are significantly higher than the other two baselines, which demonstrates the embedding-based method is effective to control the tone level and rhythm. The polishing framework on the B&T hurts a bit accuracy of tone level and rhythm, which sacrifices the tone level and rhythm constraints to obtain better semantic meaning and coherence.