Deep Reinforcement Learning with Distributional Semantic Rewards for Abstractive Summarization
Table 1: Results on Gigawords and CNN/Daily Mail for abstractive summarization. Upper rows show the results of baselines. Rouge stands for the F score of Rouge-L.
['[BOLD] Model', '[BOLD] Gigawords  [ITALIC] FBERT', '[BOLD] Gigawords  [ITALIC] PBERT', '[BOLD] Gigawords  [ITALIC] RBERT', '[BOLD] Gigawords Rouge', '[BOLD] CNN/Daily Mail  [ITALIC] FBERT', '[BOLD] CNN/Daily Mail  [ITALIC] PBERT', '[BOLD] CNN/Daily Mail  [ITALIC] RBERT', '[BOLD] CNN/Daily Mail Rouge']
For the abstractive summarization task, we test our models with different objectives on the Gigaword and CNN/Daily Mail datasets. We choose FBERT and Rouge-L as our automatic evaluation metrics. For multi-sentence summaries in CNN/ Daily Mail dataset, we calculate Rouge-L and FBERT after concatenating the original abstract sentences or their BERT contextualized embeddings. It is also expected that Rouge model will obtain worse BERTScores as simply optimizing Rouge will generate less readable sentences paulus2017deep; however, DSR model without XENT as a teacher forcing can improve the performance of pretrained model in both FBERT and Rouge-L scale.