\DATASET: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategies
Table 6: Performance of the baselines and systems on the three coverage subsets of the development set. Article-summary pairs with low coverage have reference summaries that borrow words less frequently from their texts and contain more novel words and phrases. Article-summary pairs with high coverage borrow more words from their text and include fewer novel words and phrases.
['[EMPTY]', 'Low Coverage R-1', 'Low Coverage R-2', 'Low Coverage R-L', 'Medium R-1', 'Medium R-2', 'Medium R-L', 'High Coverage R-1', 'High Coverage R-2', 'High Coverage R-L']
In addition to quantifying diversity of summarization strategies, these measures are helpful for system error analysis. We use the density measurement to understand how system performance varies when compared against references using different extractive strategies by subdividing \DATASETinto three subsets by extractiveness and evaluating using ROUGE on each. We show here a similar analysis using the remaining two measures, coverage and compression.