The Unreasonable Volatility of Neural Machine Translation Models
Table 3: BLEU scores for different models on the WMT data for translation DE↔EN.
['[EMPTY]', '[BOLD] DE-EN 2016', '[BOLD] DE-EN 2017', '[BOLD] DE-EN 2018', '[BOLD] EN-DE 2016', '[BOLD] EN-DE 2017', '[BOLD] EN-DE 2018']
In the translation experiments, we use the standard EN↔DE WMT-2017 training data (bojar-EtAl:2018:WMT1). We perform NMT experiments with two different architectures: RNN (luong:2015:EMNLP) and Transformer (vaswani2017attention). We preprocess the training data with Byte-Pair Encoding (BPE) using 32K merge operations (sennrich-haddow-birch:2016:P16-12). During inference, we use beam search with a beam size of 5.