Transfer Learning for Abstractive Summarization at Controllable Budgets
Table 5. Average completion time and accuracy by MLS, PageRank-guided Sampling (A2), convolutional Seq-to-Seq model (A3), No-Content and Full-Content setting
['[BOLD] D1', '[BOLD] Accuracy', '[BOLD] MLS  [BOLD] 0.88', '[BOLD] A2 0.55', '[BOLD] A3 0.55', '[BOLD] No-Content 0.0', '[BOLD] Full-Content  [BOLD] 0.88']
Results: We observe that the accuracy of the no-Content setting is 0 on both datasets, meaning that the questions did not contain any hint that could guide the users to the correct answer. Our results on the MSR-Narrative dataset (D1) shows that users could respond to the questions using MLS as accurately as the Full-Content setting, while completing the task more than two times faster. MLS outperformed both competing methods on dataset D1. We also noticed that one of the users were able to respond to the question correctly from MLS generated summaries while failed to do so from the original article. For the Thinking Machines dataset (D2), MLS outperformed A2 closely following A3. In terms of accuracy, out of the 6 articles selected for our experiment, MLS outperformed A3 on 4 articles and comparatively on 1. MLS outperformed A3 on 5 out of 6 articles in terms of task completion time.