Greedy Search with Probabilistic N-gram Matching for Neural Machine Translation
Table 1: Results on NIST Chinese-to-English Translation Task. AVG = average BLEU scores for test sets. The bold number indicates the highest score in the column.
['[BOLD] System', '[BOLD] Dev(MT02)', '[BOLD] MT03', '[BOLD] MT04', '[BOLD] MT05', '[BOLD] MT06', '[BOLD] AVG']
Systems We first pretrain the baseline model by maximum likelihood estimation (MLE) and then refine the model using probabilistic sequence-level objectives, including P-BLEU, P-GLEU and P-P2 (probabilistic 2-gram precision). In addition, we reproduce previous works which train the NMT model through minimum risk training (MRT) Shen et al. ) Ranzato et al. When reproducing their works, we set BLEU, GLEU and 2-gram precision as training objectives respectively and find out that GLEU yields the best performance. In the following, we only report the results with training objective GLEU. Simply training NMT model by the probabilistic 2-gram precision achieves an improvement of 1.5 BLEU points, which significantly outperforms the reinforcement-based algorithms. We also test the precision of other n-grams and their combinations, but do not notice significant improvements over P-P2. Notice that our method only changes the loss function, without any modification on model structure and training data.