Zero-Shot Dual Machine Translation
Table 1: BLEU scores on the UN corpus test set. Each line reports the BLEU scores of the corresponding translation direction. The first column refers to the phrase-based model of Ziemski et al. [20]. All NMT and Dual models are trained on 1M en-es and 1M en-fr aligned sentences, used in both directions. The NMT-S (small) model is trained additionally on 10k es-fr aligned sentences, while NMT-F (full) is trained additionally on 1M es-fr sentences. The Dual-0 model does not use any es-fr aligned data, while Dual-S is trained starting from NMT-S.
['[EMPTY]', '[BOLD] Phrase-based', '[BOLD] NMT-0', '[BOLD] NMT-S', '[BOLD] NMT-F', '[BOLD] Dual-0', '[BOLD] Dual-S']
Notice that these models are trained on the complete fully-aligned UN data, that is, 11M sentence pairs for each language pair. The second column reports the scores of our multilingual NMT baseline model (NMT-0). This is trained on 1M en-es and 1M en-fr parallel sentences. It is not trained on any es-fr data. NMT-0 performs worse than we expected between Spanish and French: 10.02 es→fr and 6.25 fr→es. In particular, we notice that NMT-0 often translates, partially, to English or the source language. The zero-shot dual system (Dual-0) starts training from the parameters of the NMT-0 model. We observe a marked improvement for the zero-shot translation directions, outperforming the baseline by 25.52 and 32.58 BLEU points for es→fr and fr→es respectively. By inspection, the translation quality seems quite reasonable. BLEU scores in the supervised translation directions degrade, as no parallel data for these pairs is seen during the incremental training. This happens for all incrementally trained models. The most important comparison is with respect to the fully supervised setting. For this purpose, we resume training the NMT-0 model with the full set of 1M es-fr parallel data. With respect to forgetting, we notice that, by never decoding to English, the NMT-F performance degrades more in this language. Interestingly, the dual model forgetting is more uniformly distributed across all original language pairs. However, NMT-F improves es→fr from 10.02 to 37.67 and fr→es, from 6.25 to 40.85 BLEU points.