GameWikiSum: a Novel Large Multi-Document Summarization Dataset
Table 4: Comparison extractive and abstractive (marked with *) models. Reported scores correspond to ROUGE-L F1 score, ROUGE-1 and ROUGE-2 recall respectively.
['[BOLD] Model', 'R-L', 'R-1', 'R-2']
LEAD-5 achieves less than 20 for ROUGE-L as well as ROUGE-1 and less than 3.5 for ROUGE-2. Taking only 3 sentences leads to even worse results: below 13 and 3 respectively. Unlike in other datasets, these results are significantly outperformed by all other extractive models but surprisingly, abstractive models perform worse on average. This demonstrates the difficulty of the task in GameWikiSum compared to \newcitenallapati2016abstractive and \newcitegraff2003.