Classical Structured Prediction Losses for Sequence to Sequence Learning
Table 6: Accuracy on Gigaword abstractive summarization in terms of F-measure Rouge-1 (RG-1), Rouge-2 (RG-2), and Rouge-L (RG-L) for token-level label smoothing, and Risk optimization of all three ROUGE F1 metrics. [T] indicates a token-level objective and [S] indicates a sequence level objectives. ABS+ refers to Rush et al. (2015), RNN MLE/MRT (Ayana et al., 2016), WFE (Suzuki and Nagata, 2017), SEASS (Zhou et al., 2017), DRGD (Li et al., 2017).
['[EMPTY]', '[BOLD] RG-1', '[BOLD] RG-2', '[BOLD] RG-L']
We optimize all three ROUGE metrics separately and find that Risk can further improve our strong baseline. but accuracy was generally lower on the validation set: RG-1 (36.59 Risk only vs. 36.67 Weighted), RG-2 (17.34 vs. 18.05), and RG-L (33.66 vs. 33.98).