Structure-Infused Copy Mechanisms for Abstractive Summarization
Table 9: Informativeness, fluency, and faithfulness scores of summaries. They are rated by Amazon turkers on a Likert scale of 1 (worst) to 5 (best). We choose to evaluate Struct+2Way+Relation (as oppose to 2Way+Word) because it focuses on preserving source relations in the summaries.
['[BOLD] System', '[BOLD] Info.', '[BOLD] Fluency', '[BOLD] Faithful.']
Linguistic quality. fluency (is the summary grammatical and well-formed?), informativeness (to what extent is the meaning of the original sentence preserved in the summary?), and faithfulness (is the summary accurate and faithful to the original?). We found that “Struct+2Way+Relation” outperforms “Struct+Input” on all three criteria. It also compares favorably to ground-truth summaries on “fluency” and “faithfulness.” On the other hand, the ground-truth summaries, corresponding to article titles, are judged as less satisfying according to human raters.