Neural Machine Translation for Bilingually Scarce Scenarios: A Deep Multi-task Learning Approach
Table 2: BLEU scores and perplexities of the baseline vs our MTL architecture with various auxiliary tasks on the full bilingual datasets.
['[EMPTY]', 'English → French Dev', 'English → French Dev', 'English → French Test', 'English → French Test', 'English → Farsi Dev', 'English → Farsi Dev', 'English → Farsi Test', 'English → Farsi Test', 'English → Vietnamese Dev', 'English → Vietnamese Dev', 'English → Vietnamese Test', 'English → Vietnamese Test']
It can be seen that the performance of multi-task learning models are better than Baseline 1 (only MT task). This confirms that adding auxiliary tasks helps to increase the performance of the machine translation task. We have further added adversarial training to ensure the shared representation learned by the encoder is not contaminated by the task-specific information. The experiments show that adversarial training leads to further gains in MTL translation quality, except when translating into Farsi. We speculate this is due to the low quality of NMT for Farsi, where updating shared parameters with respect to the entropy of discriminator’s predicted distribution may negatively affect the model.