Analyzing Sentence Fusion in Abstractive Summarization
Table 3: Results for each merging method. Concatenation has high faithfulness, grammaticality, and coverage, while Replacement and Other have much lower scores.
['[BOLD] System', '[BOLD] Faithful', '[BOLD] Grammatical', '[BOLD] Coverage']
Does the way sentences are fused affect their faithfulness and grammaticality? Grammaticality is relatively high for all merging categories. Coverage is also high for balanced/imbalanced concatenation and replacement, meaning the heuristic works succesfully for these forms of sentence merging. It does not perform as well on the Other category. This is understandable, since sentences formed in a more complex manner will be harder to identify using simple word overlap. Faithfulness has a similar trend, with summaries generated using concatenation being more likely to be faithful to the original article. This may explain why PG is the most faithful of the systems, while being the simplest—it uses concatenation more than any of the other systems. We believe more effort can be directed towards improving the more complex merging paradigms, such as entity replacement.