Cross-lingual Pre-training Based Transfer for Zero-shot Neural Machine Translation
Table 2: Results on Europarl test sets. Three pivot settings are conducted in our experiments. In each setting, the left column presents the zero-shot performances (source→target), and the right column denotes the performances in the supervised parent model direction (pivot→target).
['Europarl Direction', 'Fr → En → Es Fr → Es', 'Fr → En → Es En → Es', 'De → En → Fr De → Fr', 'De → En → Fr En → Fr', 'Ro → En → De Ro → De', 'Ro → En → De En → De']
We compare our approaches with related approaches of pivoting, multilingual NMT (MNMT) [johnson2017google], and cross-lingual transfer without pretraining [kim2019effective]. The results show that our approaches consistently outperform other approaches across languages and datasets, especially surpass pivoting, which is a strong baseline in the zero-shot scenario that multilingual NMT systems often fail to beat [johnson2017google, al2019consistency, arivazhagan2019missing]. Pivoting translates source to pivot then to target in two steps, causing inefficient translation process. Our approaches use one encoder-decoder model to translate between any zero-shot directions, which is more efficient than pivoting. Regarding the comparison between transfer approaches, our cross-lingual pretraining based transfer outperforms transfer method that does not use pretraining by a large margin.