Toward Abstractive Summarization Using Semantic Representations
Table 5: Subgraph prediction and summarization (to bag of words) results on test set. Gold-standard AMR annotations are used for model training in all conditions. “+ Expand” means the result is obtained using source graph with expansion; edge performance is measured ignoring labels.
['[EMPTY]', '[EMPTY]', '[BOLD] Subgraph Prediction  [BOLD] Nodes', '[BOLD] Subgraph Prediction  [BOLD] Nodes', '[BOLD] Subgraph Prediction  [BOLD] Nodes', '[BOLD] Subgraph Prediction  [BOLD] Edges', '[BOLD] Summarization  [BOLD] ROUGE-1', '[BOLD] Summarization  [BOLD] ROUGE-1', '[BOLD] Summarization  [BOLD] ROUGE-1']
Gold-standard AMR annotations are used for model training in all conditions. During testing, we apply the trained model to source graphs constructed using either gold-standard or JAMR parses. In all of these experiments, we use the number of edges in the gold-standard summary graph to fix the number of edges in the predicted subgraph, allowing direct comparison across conditions. “+ Expand” means the results are obtained using the expanded source graph. We find that graph expansion only marginally affects system performance. Graph expansion slightly hurts the system performance on edge prediction. For example, using ramp loss with JAMR parser as input, we obtained 50.7% and 19.0% for node and edge prediction with graph expansion; 51.5% and 20.0% without edge expansion. On the other hand, it increases the oracle performance by a large margin. This suggests that with more training data, or a more sophisticated model that is able to better discriminate among the enlarged output space, graph expansion still has promise to be helpful.