Abstract Meaning Representation for Multi-Document Summarization
Table 2: Summarization results on DUC-04 and TAC-11 datasets. We compare the AMR summarization framework (AMRSumm-*) with both extractive (ext-*) and abstractive (abs-*) summarization systems.
['[EMPTY]', '[BOLD] System', '[BOLD] ROUGE-1  [BOLD] P', '[BOLD] ROUGE-1  [BOLD] R', '[BOLD] ROUGE-1  [BOLD] F', '[BOLD] ROUGE-2  [BOLD] P', '[BOLD] ROUGE-2  [BOLD] R', '[BOLD] ROUGE-2  [BOLD] F', '[BOLD] ROUGE-SU4  [BOLD] P', '[BOLD] ROUGE-SU4  [BOLD] R', '[BOLD] ROUGE-SU4  [BOLD] F']
In particular, R-1, R-2, and R-SU4 respectively measure the overlap of unigrams, bigrams, and skip bigrams (up to 4 words) between system and reference summaries. Our AMR summarization framework outperforms all other abstractive systems with respect to R-1 and R-SU4 F-scores on both DUC-04 (trained on DUC-03) and TAC-11 (trained on TAC-09,10) datasets. “AMRSumm-VSM” further produces the highest R-2 F-scores. When it transforms a summary graph to text, there can be multiple acceptable realizations (e.g., “I hit my hand on the table” or “My hand hit the table”) and the one chosen by the AMR generator may not always be the same as the source text. Because abstractive systems are expected to produce novel words, they may yield slightly inferior results to the best extractive system (LexRank). Similar findings are also reported by Nallapati et al. \shortciteNallapati:2017 and See et al.