Regularizing Neural Machine Translation by Target-bidirectional Agreement
Table 4: Translation performance of our method on WMT validation sets during training process. “Iteration 0” denotes baseline Transformer model.
['[EMPTY]', 'English-German', 'Chinese-English']
We further investigate the impact of our joint training algorithm during the whole training process. For each iteration, we train NMT models until the performance on the development set does not increase. We find that more iterations can lead to better evaluation results consistently and 2 iterations are enough to reach convergence in our experiments. However, further iterations cannot bring noticeable translation accuracy improvements but more training time. For training cost, since our method is based on pre-trained models, the entire training time is almost 2 times the original MLE training.