Training on Synthetic NoiseImproves Robustness to Natural Noise in Machine Translation
Table 2: The performance of a machine translation model on the IWSLT 2016 task when different amounts of natural orthographical errors are introduced. Noise Probability is the probability of a test-data token to be injected with natural noise, while the Noised Tokens column reflects the number of tokens that were actually distorted in practice; not every word in the vocabulary has a corresponding misspelling. Training on synthetic noise significantly increases robustness in scenarios where large amounts of spelling mistakes are encountered.
['[BOLD] Dataset', '[BOLD] Noise Probability', '[BOLD] Noised Tokens', '[BOLD] BLEU  [BOLD] Clean Training Data', '[BOLD] BLEU  [BOLD] + Synthetic Noise', '[BOLD] BLEU Δ']
As observed in prior art Heigold et al. However, training on our synthetic noise cocktail greatly improves performance, regaining between 20% (Czech) and 50% (German) of the BLEU score that was lost to natural noise. Moreover, the negative effects of training on synthetic noise seem to be limited to both negative and positive fluctuations that are smaller than 1 BLEU point.