Evaluating the Supervised and Zero-shot Performance of Multi-lingual Translation Models
Table 6: Out of 110 pivot translation tasks, how many failed the language identification check?
['[EMPTY]', '[BOLD] Prepend', '[BOLD] Emb', '[BOLD] Dec', '[BOLD] Attn']
For all models except Emb only ru→kk→ru fi→lt→fi, and zh→gu→zh failed this test, but for the Emb model 31 of the 110 translation directions failed (see This result indicates that models which use language embeddings may have a more "fuzzy" representation of the output task, and are much more prone to copying than other approaches to multi-lingual MT.