Capturing document context inside sentence-level neural machine translation models with self-training
Table 3: Results on WMT’19 Chinese-English evaluation sets. The first row shows the performance of the Transformer Big model by Xia et al. (2019). All models were trained without additional monolingual data and without pretraining. ST: self-training.
['Method', 'Architecture', 'Model', 'Valid17', 'Test17', 'Test18', 'Test19', 'ΔBLEU']
Compared to the NIST dataset our self-training procedure shows an improvement of 0.1 BLEU over a sentence-level baseline model. Oracle self-training outperforms sentence-level baselines by a significant margin of 2.5 BLEU. We hypothesize that such a large gap between performance of oracle and non-oracle self-training is due to the more challenging nature of the WMT dataset which is reflected in the worse performance of sentence-level baseline on WMT compared to NIST. We investigate this claim by measuring the relationship between BLEU achieved by self-training and the relative quality of the sentence-level model on the NIST dataset. This implies that we can expect a larger improvement from applying self-training as we improve the sentence-level model on the WMT dataset. Preliminary experiments on training back-translated models didn’t improve results on the WMT dataset. We leave further investigation of ways to improve the sentence-level model on the WMT dataset for future work.