Abstract Meaning Representation for Multi-Document Summarization
Table 3: Summarization results of the AMR-Summ framework on the DUC-04 dataset. “JAMR” uses the JAMR parser [Flanigan et al.2014] to produce AMR graphs for source sentences, while “CAMR” uses the CAMR parser [Wang et al.2015b]. “Oracle” results are obtained by performing only cost-based decoding.
['[EMPTY]', '[BOLD] Approach', '[BOLD] JAMR  [BOLD] P', '[BOLD] JAMR  [BOLD] R', '[BOLD] JAMR  [BOLD] F', '[BOLD] CAMR  [BOLD] P', '[BOLD] CAMR  [BOLD] R', '[BOLD] CAMR  [BOLD] F', '[BOLD] (Oracle) CAMR  [BOLD] P', '[BOLD] (Oracle) CAMR  [BOLD] R', '[BOLD] (Oracle) CAMR  [BOLD] F']
We observe that the quality of AMR parsers has an impact on summarization performance. In particular, JAMR reports 58% F-score and CAMR reports 63% F-score for parsing sentences. There remains a significant gap between the current parser performance and the best it can achieve. Consequently, we notice that there is a gap of 1∼2 points in terms of ROUGE scores when comparing summaries produced using the two parsers. We notice that source sentence selection strategies making use of reference summary sentences produces better results than ‘AMRSumm-Clst.’ Using oracle decoding further boosts the summarization performance by 7-10% for R-1 F-score and 2-5% for R-2 F-score.