Evaluating Robustness to Input Perturbationsfor Neural Machine Translation
Table 3: BLEU, robustness (in percentage), and consistency scores of different subword segmentation methods on original and perturbed test sets. We report mean and standard deviation using bootstrap resampling (Koehn, 2004). Subword regularization makes NMT models more robust to input perturbations.
['[EMPTY]', 'Model', 'BLEU EN→DE (newstest2019)', 'ROBUST EN→DE (newstest2019)', 'CONSIS EN→DE (newstest2019)', 'BLEU DE→EN (newstest2019)', 'ROBUST DE→EN (newstest2019)', 'CONSIS DE→EN (newstest2019)']
One major difference from previous work is the size of the training data, which is much larger in our experiments – subword regularization is presumably preferable on low-resource settings. We conduct more comprehensive analysis on the correlation between them, and we collect additional data points by varying the noise level of both perturbations. Specifically, we use the following word misspelling probabilities: {0.05,0.1,0.15,0.2} and the following sentence case-changing probability values: {0.3,0.5,0.7,0.9}.