\DATASET: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategies
Table 7: Performance of the baselines and systems on the three compression subsets of the development set. Article-summary pairs with low compression have longer reference summaries with respect to their texts. Article-summary pairs with high compression have shorter reference summaries with respect to their texts.
['[EMPTY]', 'Low Compression R-1', 'Low Compression R-2', 'Low Compression R-L', 'Medium R-1', 'Medium R-2', 'Medium R-L', 'High Compression R-1', 'High Compression R-2', 'High Compression R-L']
In addition to quantifying diversity of summarization strategies, these measures are helpful for system error analysis. We use the density measurement to understand how system performance varies when compared against references using different extractive strategies by subdividing \DATASETinto three subsets by extractiveness and evaluating using ROUGE on each. We show here a similar analysis using the remaining two measures, coverage and compression.