Transfer Learning for Abstractive Summarization at Controllable Budgets
Table 4. Experimental results on the MSR-Narrative (D1) and Thinking Machines (D2) dataset by MLS (highlighted column) and MLS†. The best performance achieved for each metric is shown in boldface
['[BOLD] Dataset', '[BOLD] Metric', '[BOLD] Budget = 1/32  [BOLD] MLS', '[BOLD] Budget = 1/32  [BOLD] MLS†', '[BOLD] Budget = 1/16  [BOLD] MLS', '[BOLD] Budget = 1/16  [BOLD] MLS†', '[BOLD] Budget = 1/8  [BOLD] MLS', '[BOLD] Budget = 1/8  [BOLD] MLS†', '[BOLD] Budget = 1/4  [BOLD] MLS', '[BOLD] Budget = 1/4  [BOLD] MLS†', '[BOLD] Budget = 1/2  [BOLD] MLS', '[BOLD] Budget = 1/2  [BOLD] MLS†']
Effect of pretraining: To isolate gains in end-to-end performance due to pretraining on the CNN-DailyMail dataset, we compare against MLS†, a baseline identical to MLS without any pretraining. We observed significant improvement over MLS†, thus establishing that pretraining helps improve the quality of our summaries. The effect on performance is apparent for all compression budgets in our test suite.