Denoising based Sequence-to-Sequence Pre-training for Text Generation
Table 4: ROUGE scores for Gigaword dataset.
['System', 'ROUGE 1', 'ROUGE 2', 'ROUGE L']
Results for Gigaword The Gigaword dataset is much larger than CNN/Daily Mail, and this enables “PoDA w/o pre-training” to have competitive performance even without pre-training. We can see that PoDA performs favorably over the state-of-the-art Re3Sum method, which utilizes unlabeled text corpora by first retrieving and then rewriting relevant snippets. With only 103 training examples, PoDA reaches a reasonably good performance comparable to ABS+ (an attention-based system trained on nearly 4 million examples). With more labeled data available, the performance gap between “PoDA