Sentence Simplification: Challenges for Encoder-Decoder Models
Table 5: Confusion matrix for choosing to change or copy a word. The rows are the actions chosen by the S4+gv+bce model when fed ground-truth alignments. The columns are the ground truth actions.
['[BOLD] Generated', '[BOLD] Ground Truth Copy Word', '[BOLD] Ground Truth Change Word']
S4+gv+bce frequently chooses to copy words, mirroring the high similarity between ground-truth sentences. However, in only 32% of cases does S4+gv+bce correctly choose to change a word. Even when S4+gv+bce correctly decides to change a word, it only chooses the same word as the ground-truth 46% of the time. Word replacement itself does not perform well. Better word replacement may be archived with significantly more training data, though new ideas seem necessary for further improvement in the more common case of data scarcity.