Non-Monotonic Sequential Text Generation
Table 4: Word Reordering results on Persona-Chat, reported according to Bleu score, F1 score, and percent exact match.
['[BOLD] Oracle', '[BOLD] Validation  [BOLD] Bleu', '[BOLD] Validation  [BOLD] F1', '[BOLD] Validation  [BOLD] EM', '[BOLD] Test  [BOLD] Bleu', '[BOLD] Test  [BOLD] F1', '[BOLD] Test  [BOLD] EM']
The uniform and annealed policies outperform the left-right policy in F1 score (0.96 and 0.95 vs. 0.903). The policy trained using the annealed oracle also matches the left-right policy’s performance in terms of Bleu score (46.0 vs. 46.3) and exact match (0.212 vs. 0.208). The model trained with the uniform policy does not fare as well on Bleu or exact match. See Appendix Figure 6 for example predictions.