Parameter Sharing Methods for Multilingual Self-Attentional Translation Models
Table 3: BLEU scores for different models for one-to-many translation task. NS: No Sharing corresponds to the bilingual models when the two language pairs are trained independently; FS: Full Sharing means one model is used for the translation of all the language pairs; PS: Partial Sharing means that the embedding, encoder, decoder’s key, and value weights are shared between the two models.
['Method', 'En→De+Tr →De', 'En→De+Tr →Tr', 'En→De+Ja →De', 'En→De+Ja →Ja', 'En→Ro+Fr →Ro', 'En→Ro+Fr →Fr', 'En→De+Nl →De', 'En→De+Nl →Nl', '[ITALIC] params ×106']
Wu et al. First, we note that the BLEU scores of the Transformer model are always better than the GNMT model by a significant margin for both bilingual (no sharing) and multilingual (full sharing) translation tasks. This reflects that the Transformer model is well-suited for both multilingual and bilingual translation tasks compared with the GNMT model. We also surprisingly note that the GNMT fully shared model is able to consistently obtain higher BLEU scores compared with its bilingual version irrespective of which families the target languages belong to.