Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation
Table 7: Zero-short translation quality for ROBt under different settings. “100-to-100”: the setting used in the above experiments; we set T to all target languages. “6-to-6”: T only includes the zero-shot languages in the test set. We employ 6-layer Transformer with LaLn and LaLt for experiments.
['Setting', 'BLEU [ITALIC] zero']
Impact of T on ROBt. ROBt heavily relies on T, the set of target languages considered, to distribute the modeling capacity on zero-shot directions. To study its impact, we provide a comparison by constraining T to 6 languages in the zero-shot test set. By narrowing T, more capacity is scheduled to the focused languages, which results in performance improvements. But the small scale of this improvement suggests that the number of zero-shot directions is not ROBt’s biggest bottleneck.