Facet-Aware Evaluation for Extractive Summarization
Table 3: Performance comparison of extractive methods under ROUGE F1 and Facet-Aware Recall (FAR).
['[BOLD] Method', '[BOLD] ROUGE-1', '[BOLD] ROUGE-2', '[BOLD] ROUGE-L', '[BOLD] FAR']
Results on Neural Extractive Methods. The observations on ROUGE Precision/Recall are similar. We provide them as well as more comparative analysis under facet-aware evaluation in Sec. For facet coverage, the upper bound of FAR by extracting 3 sentences (Oracle, given the ground-truth FAMs) is 84.8, much higher than all the compared methods. The best performing extractive method under FAR is UnifiedSum(E), which indicates that it covers the most facets semantically.