Sentence Centrality Revisited for Unsupervised Summarization
Table 4: Results of QA-based evaluation on NYT, CNN/Daily Mail, and TTNews. We compute a system’s final score as the average of all question scores. Systems statistically significant from PacSum are denoted with an asterisk * (using a one-way ANOVA with posthoc Tukey HSD tests; p<0.01).
['[BOLD] Method', '[BOLD] NYT', '[BOLD] CNN+DM', '[BOLD] TTNews']
Oracle’s performance is below 100, indicating that extracting sentences by maximizing ROUGE fails in many cases to select salient content, capturing surface similarity instead. PacSum significantly outperforms Lead but is worse than Oracle which suggests there is room for further improvement. Interestingly, PacSum performs on par with Refresh (the two systems are not significantly different).