Facet-Aware Evaluation for Extractive Summarization
Table 6: Performance of sentence regression approaches regarding support sentence discovery. High precision and low recall are often observed.
['[BOLD] Method', '[BOLD] Precision', '[BOLD] Recall', '[BOLD] F1']
Results on Support Sentence Discovery. We first evaluate sentence regression with its original function, i.e., creating extractive labels (finding support sentences). We merge the support groups of each sample and calculate precision and recall (i.e., SAR). The relatively low recall suggests that simply finding one support sentence for each facet as most existing approaches do would miss plenty of salient sentences, which could possibly worsen the models trained on such labels since the models would treat missed support sentences as unimportant ones. On the bright side, many sentence regression approaches achieve high precision. For instance, 90.0% document sentences labeled positive by narayan-etal-2018-ranking indeed contain salient information. This is to some extent explainable as ROUGE captures lexical overlap and as we have shown, there are many copy-and-paste reference summaries in CNN/Daily Mail.