Data Diversification: An Elegant Strategy For Neural Machine Translation
Table 9: BLEU scores for models with and without back-translation (BT) on the IWSLT’14 English-German (En-De), German-English (De-En) and WMT’14 En-De tasks. Column |D| shows the total data used in back-translation compared to the original parallel data.
['[BOLD] Task', '[BOLD] No back-translation  [BOLD] Baseline', '[BOLD] No back-translation  [BOLD] Ours', '[BOLD] With back-translation | [ITALIC] D|', '[BOLD] With back-translation  [BOLD] Baseline', '[BOLD] With back-translation  [BOLD] Ours']
Our method is also complementary to back-translation (BT) (backtranslate_sennrich-etal-2016-improving). To demonstrate this, we conducted experiments on the IWSLT’14 En-De and De-En tasks with extra monolingual data extracted from the WMT’14 En-De corpus. In addition, we also compare our method against the back-translation baseline in the WMT’14 En-De task with extra monolingual data from News Crawl 2009. We use the big Transformer as the final model in all our back-translation experiments. Further details of these experiments are provided in the Appendix. However, using our data diversification strategy with such monolingual data boosts the performance further with additional +1.0 BLEU over the back-translation baselines.