Multilingual Denoising Pre-training for Neural Machine Translation
Table 12: Back-Translation v.s. Language Transfer for Unsupervised MT. We present the best transferring scores together with the pairs transferred from.
['[BOLD] Pairs  [BOLD] Ro→En', '[BOLD] BT 30.5', '[BOLD] Transfer  [BOLD] Cs→En', '[BOLD] Transfer 23.0', '[BOLD] Combined  [BOLD] 33.9']
S5SS2SSS0Px4 The results are also mixed. If there exists high quality (similar languages) bi-text data, or translating between dissimilar pairs, language transfer is able to beat the conventional methods with BT. Furthermore, we also show promising results for combining these two techniques. In such cases, we start from the best transferred model and apply (iterative) BT on the same monolingual corpus used in pre-training. For all pairs, we see improvements by combining both techniques.