Exploring Explainable Selection to Control Abstract Generation
Table 7: QA-based and criteria-based human evaluation. ∗ mark indicates the improvements from the baselines to the ESCA-BERT are statistically significant using a paired t-test (p<0.05). Gold summary was not included in QA evaluation. ♭ means unnecessary value.
['Models', 'QA', 'Criteria Infor.', 'Criteria Nov.', 'Criteria Rel.']
In the first block of criteria ranking, 5 systems were simultaneously ranked. The gold summary set the upper bound except for the relevance. Unsurprisingly, since the gold summaries of CNN/DailyMaily are mostly top sentences in the articles, their relevance can not be guaranteed. We also found the ESCA-BERT produced the most popular summaries. In the second block, the ESCA with novelty and relevance control were also evaluated together with the bottom-up and the original ESCA. The ranking difference slightly varies, but it obviously proves that the ESCA under novelty or relevance control gained the corresponding highest rank.