Building a Neural Machine Translation SystemUsing Only Synthetic Parallel Data
Table 2: Translation results (BLEU) for Fr ↔ De experiments. The notation * denotes the synthetic part of the parallel corpus. The highest BLEU for each set is bold-faced.
['Corpus', 'Fr → De newstest2011', 'Fr → De newstest2012', 'Fr → De newstest2013', 'De → Fr newstest2011', 'De → Fr newstest2012', 'De → Fr newstest2013']
A Comparison between Pivot-based Approach and Back-translation Before we choose the pivot language-based method for data synthesis , we conduct a preliminary experiment analyzing both pivot-based and direct back-translation. On the newstest 2012/2013 sets, the synthetic corpus generated using the pivot approach showed higher BLEU (19.11 / 20.45) than the back-translation counterpart (18.23 / 19.81) when used in training a De→Fr NMT model. Although the back-translation method has been effective in many studies Sennrich et al. This is due to the poor quality of the back-translation model built from the limited source-to-target parallel corpus. Instead, one can utilize abundant pivot-to-target parallel corpora by using a rich-resource language as the pivot language. This consequently improves the reliability of the quality of baseline translation models used for generating synthetic corpora. , we find that the bias of the synthetic examples in pseudo parallel corpora brings imbalanced quality in the bidirectional translation tasks. Given that the source- and target-originated classification of a specific synthetic corpus is reversed depending on the direction of the translation, the overall results imply that the target-originated corpus for each translation task outperforms the source-originated data. The preference of target-originated synthetic data over the source-originated counterparts was formerly investigated in SMT by Lambert et al., Lambert et al. In NMT, it can be explained by the degradation in quality in the source-originated data owing to the erroneous target language model formed by the synthetic target sentences. In contrast, we observe that PSEUDOmix not only produces balanced results for both Fr→De and De→Fr translation tasks but also shows the best or competitive translation quality for each task.