BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension
Table 6: The performance (BLEU) of baseline and BART on WMT’16 RO-EN augmented with back-translation data. BART improves over a strong back-translation (BT) baseline by using monolingual English pre-training.
['[EMPTY]', 'RO-EN']
We also evaluated performance on WMT16 Romanian-English, augmented with back-translation data from Sennrich et al. We compare our results against a baseline Transformer architecture Vaswani et al. We show the performance of both steps of our model in the fixed BART and tuned BART rows. For each row we experiment on the original WMT16 Romanian-English augmented with back-translation data. We use a beam width of 5 and a length penalty of α=1. Preliminary results suggested that our approach was less effective without back-translation data, and prone to overfitting—future work should explore additional regularization techniques.