Generative Neural Machine Translation
Table 1: Test set BLEU scores on pure translation for models trained with varying amounts of paired sentences.
['Paired data', 'System', 'EN→ES', 'ES→EN', 'EN→FR', 'FR→EN', 'ES→FR', 'FR→ES']
When trained with 40K and 400K paired sentences, GNMT has a small advantage over VNMT in terms of BLEU scores across all language pairs. However, both models tend to overfit on these relatively small amounts of paired data. As a result, GNMT-Multi achieves much higher BLEU scores with both 40K and 400K paired sentences, due to the parameter sharing between languages. Adding monolingual data produces yet another significant increase in BLEU scores. In fact, GNMT-Multi-SSL trained with only 400K paired sentences achieves performance comparable with GNMT trained with 4M paired sentences. Even with 4M paired sentences, adding monolingual data is helpful, with GNMT-Multi-SSL outperforming the other models.