Better Rewards Yield Better Summaries: Learning to Summarise Without References
Table 5: Performance of ExtAbsRL with different reward functions, measured in terms of ROUGE (center) and human judgements (right). Using our learned reward yields significantly (p=0.0057) higher average human rating. “Pref%”: in how many percentage of documents a system receives the higher human rating.
['Reward', 'R-1', 'R-2', 'R-L', 'Human', 'Pref%']
Table 5 compares the ROUGE scores of using different rewards to train the extractor in ExtAbsRL (the abstractor is pre-trained, and is applied to rephrase the extracted sentences). Again, when ROUGE is used as rewards, the generated summaries have higher ROUGE scores. [CONTINUE] It is clear from Table 5 that using the learned reward helps the RL-based system generate summaries with significantly higher human ratings.