Dataset for Automatic Summarization of Russian News
Table 2: Average % of novel n-grams
['[EMPTY]', 'Train', 'ha Val', 'ha Test']
To evaluate the dataset’s bias towards extractive or abstractive methods, we measured how many novel n-grams summaries have. This number decreases to 58% if we consider different word forms and calculate it on lemmatized bi-grams. Although we can not directly compare these numbers with CNN/DailyMail or any other English dataset as this statistic is heavily language-dependent, but we should state that it is 53% for CNN/DailyMail and 83% for XSum. From this, we can conclude that the bias towards extractive methods can exist.