Question Answering as an Automatic Evaluation Metric for News Article Summarization
Table 1: Pearson Correlation of ROUGE and APES against Pyramid and Responsiveness on summary level. Statistically significant differences are marked with *.
['[EMPTY]', 'ROUGE-1', 'ROUGE-2', 'ROUGE-L', 'ROUGE-SU', 'APES']
We follow the work of Louis and Nenkova and compare input level APES scores with manual Pyramid and Responsiveness scores provided in the AESOP task. In Input level, correlation is computed for each summary against its manual score. In contrast, system level reports the average score for a summarization system over the entire dataset.