Summary Level Training of Sentence Rewriting for Abstractive Summarization
Table 5: Results of human evaluation.
['[BOLD] Models', '[BOLD] Relevance', '[BOLD] Readability', '[BOLD] Total']
We also conduct human evaluation to ensure robustness of our training procedure. We measure relevance and readability of the summaries. Relevance is based on the summary containing important, salient information from the input article, being correct by avoiding contradictory/unrelated information, and avoiding repeated/redundant information. Readability is based on the summarys fluency, grammaticality, and coherence. 2, 1 and 0 points were given according to the ranking. The models were anonymized and randomly shuffled. Following previous work, the input article and ground truth summaries are also shown to the human participants in addition to the three model summaries. In terms of readability, there was no noticeable difference.