A Pretrained Unsupervised Summarization Model with Theme Modeling and Denoising
Table 2: ROUGE F1 scores on NYT and CNN/DM datasets. R1/R2/RL stands for ROUGE-1/ROUGE-2/ROUGE-L respectively. Best results in each unsupervised category is in bold. Results of other models are obtained from original papers or running open-sourced software.
['Model', 'CNN/DM R1', 'CNN/DM R2', 'CNN/DM RL', 'NYT R1', 'NYT R2', 'NYT RL']
Note that ROUGE metric prefers extractive systems that preserve original phrasing Considering this factor, TED achieves results that are competitive with unsupervised extractive baselines and surpasses all unsupervised abstractive models. We use the same training, validation and test split (287,227/13,368/11,490) as other baselines. Similar to See et al. TED with a larger model size (10L8H) outperforms all unsupervised abstractive methods and compares favorably with unsupervised extractive baselines. Note that TED outperforms GTP-2, a powerful transformer-based language generation model pretrained on large scale webpage textual data, by siginificant margins. Again, TED further improves upon pretrained models on both 10L8H and 4L4H configurations.