Character-based NMT with Transformer
5,000
['train \\test', 'clean', 'all', 'delete', 'insert', 'replace', 'switch', 'avg']
Adding noise helps. Training on similar type of noisy data improves performance for all vocabularies. By training on similar kinds of noise in the training data, we are able to robustify BPE models to the same level as character level models without sacrificing too much performance on the clean test set. Effect on clean data. However, in the case of BPE 30 000, training on noisy data significantly boosted performance (eg, improvement of 6 BLEU for DE-EN and 1.7 for EN-DE when training with delete and testing on clean data). We hypothesize that the increased diversity of tokens during training (due to the presence of noise) acts as a regularizer boosting performance on the test set.