Leveraging Graph to Improve Abstractive Multi-Document Summarization
Table 5: Ranking results of system summaries by human evaluation. 1 is the best and 5 is the worst. The larger rating denotes better summary quality. R.B. and G.S. are the abbreviations of RoBERTa and GraphSum, respectively. ∗ indicates the overall ratings of the corresponding model are significantly (by Welch’s t-test with p<0.01) outperformed by our models GraphSum and GraphSum+RoBERTa.
['[BOLD] Model', '1', '2', '3', '4', '5', 'Rating']
Both the percentage of ranking results and overall ratings are reported. The results demonstrate that GraphSum and GraphSum+RoBERTa are able to generate higher quality summaries than other models. Specifically, the summaries generated by GraphSum and GraphSum+RoBERTa usually contains more salient information, and are more fluent and concise than other models. The human evaluation results further validates the effectiveness of our proposed models.