Data Diversification: An Elegant Strategy For Neural Machine Translation
Table 10: WMT’14 English-German (En-De) and English-French (En-Fr) diversity performances in BLEU and Pairwise-BLEU scores. Lower Pairwise-BLEU means more diversity, higher BLEU means better quality.
['[BOLD] Method', '[BOLD] Pairwise-BLEU  [BOLD] En-De', '[BOLD] Pairwise-BLEU  [BOLD] En-Fr', '[BOLD] BLEU  [BOLD] En-De', '[BOLD] BLEU  [BOLD] En-Fr']
By just training multiple models with different seeds, their translations from the training set yield only 14% and 22% duplicates for En-De and En-Fr. These results may be surprising as we might expect more duplicates. To evaluate the diversity the teacher models used in data diversification, we compare them with the BLEU/Pairwise-BLEU benchmark proposed by mixture_model_nmt_shen2019. Specifically, we use our forward models trained on WMT’14 English-German and English-French and measure the BLEU and Pairwise-BLEU scores in the provided testset. As it can be seen in En-De experiments, our method is less diverse than the mixture of experts method (hMup) with 57.1 versus 50.2 Pairwise-BLEU. However, translations of our method is of better quality (69.5 BLEU), which is very close to human performance. The same conclusion can be derived from the WMT’14 English-French experiments.