Multi-Reward Reinforced Summarization with Saliency and Entailment
Table 4: Abstractiveness: novel n-gram percentage.
['Models', '2-gram', '3-gram', '4-gram']
In order to measure the abstractiveness of our models, we followed the ‘novel n-gram counts’ approach suggested in \newcitesee2017get. First, we found that all our reward-based RL models have significantly (p<0.01) more novel n-grams than our cross-entropy baseline (see Next, the Entail-reward model ‘maintains’ stat. equal abstractiveness as the ROUGE-reward model, likely because it encourages rewriting to create logical subsets of information, while the ROUGESal-reward model does a bit worse, probably because it focuses on copying more salient information (e.g., names). Compared to previous work See et al.