Minimum Risk Training for Neural Machine Translation
Table 2: Effect of loss function on the Chinese-English validation set.
['criterion', 'loss', 'BLEU', 'TER', 'NIST']
As our approach is capable of incorporating evaluation metrics as loss functions, we investigate the effect of different loss functions on BLEU, TER and NIST scores on the Chinese-English validation set. Note that the loss functions are all defined at the sentence level while evaluation metrics are calculated at the corpus level. This discrepancy might explain why optimizing with respect to sTER does not result in the lowest TER on the validation set. As −sBLEU consistently improves all evaluation metrics, we use it as the default loss function in our experiments.