Pivot-based Transfer Learning for Neural Machine Translationbetween Non-English Languages
Table 3: Pivot adapter variations (German→Czech). All results are tuned with source-target parallel data.
['Adapter Training', 'newstest2013 Bleu [%]', 'newstest2013 Ter [%]']
The row “None” shows that a randomly initialized linear layer already guides the pre-trained encoder/decoder to harmonize with each other. Of course, when we train the adapter to map source encoder outputs to pivot encoder outputs, the performance gets better. For compressing encoder outputs over positions, average-pooling is better than max-pooling. We observed the same trend in the other test set and in French→German.