Deep-speare: A joint neural model of poetic language, meter and rhyme
Table 2: Component evaluation for the language model (“Ppl” = perplexity), pentameter model (“Stress Acc”), and rhyme model (“Rhyme F1”). Each number is an average across 10 runs.
['[BOLD] Model', '[BOLD] Ppl', '[BOLD] Stress Acc', '[BOLD] Rhyme F1']
Encouragingly, we see that the incorporation of character encodings and preceding context improves performance substantially, reducing perplexity by almost 10 points from LM to LM∗∗. The inferior performance of LM∗∗-C compared to LM∗∗ demonstrates that our approach of processing context with recurrent networks with selective encoding is more effective than convolutional networks. The full model LM∗∗+PM+RM, which learns stress and rhyme patterns simultaneously, also appears to improve the language model slightly. LM∗∗+PM+RM performs competitively, and informal inspection reveals that a number of mistakes are due to dictionary errors. The y-axis represents the ten stresses of the iambic pentameter, and x-axis the characters of the sonnet line (punctuation removed). The attention network appears to perform very well, without any noticeable errors. The only minor exception is lovely in the second line, where it predicts 2 stresses but the second stress focuses incorrectly on the character e rather than y. Additional heatmaps for the full sonnet are provided in the supplementary material. The rhyme model performs very strongly at F1 > 0.90, well above both baselines. Rhyme-EM performs poorly because it operates at the word level (i.e. it ignores character/orthographic information) and hence does not generalise well to unseen words and word pairs.