A Large-Scale Multi-Document Summarization Dataset from the Wikipedia Current Events Portal
Table 5: Evaluation results on test set.
['[BOLD] F-score Method', '[BOLD] F-score R1', '[BOLD] F-score R2', '[BOLD] F-score RL']
The supervised methods TSR and BertReg show advantages over unsupervised methods, but not by a large margin, which poses an interesting challenge for future work. The high extractive bounds defined by Oracle (Single) suggest that identifying important documents before summarization can be useful in this dataset. The dataset does not favor lead summaries: Random Lead is of low quality, and Lead Oracle has relatively low F-scores (although very high Recall). The Submodular + Abs heuristic for applying a pre-trained abstractive model does not perform well.