Bridging the Gap between Training and Inference for Neural Machine Translation
Table 1: Case-insensitive BLEU scores (%) on Zh→En translation task. “‡”, “†”, “⋆” and “∗” indicate statistically significant difference (p<0.01) from RNNsearch, SS-NMT, MIXER and Transformer, respectively.
['[BOLD] Systems', '[BOLD] Architecture', '[BOLD] MT03', '[BOLD] MT04', '[BOLD] MT05', '[BOLD] MT06', '[BOLD] Average']
Shen et al. Zhang et al. model distortion to enhance the attention model. Compared with them, our baseline system RNNsearch 1) outperforms previous shallow RNN-based NMT system equipped with the coverage model Tu et al. We hope that the strong shallow baseline system used in this work makes the evaluation convincing. We also compare with the other two related methods that aim at solving the exposure bias problem, including the scheduled sampling Bengio et al. (SS-NMT) and the sentence-level training Ranzato et al. (MIXER). While our approach OR-NMT can outperform the baseline system RNNsearch and the competitive comparison systems by directly incorporate the sentence-level oracle and noise perturbations for relieving the overcorrection problem. Particularly, our OR-NMT significantly outperforms the RNNsearch by +2.36 BLEU points averagely on four test datasets. Comparing with the two related models, our approach further gives a significant improvements on most test sets and achieves improvement by about +1.2 BLEU points on average.