On The Alignment Problem In Multi-Head Attention-Based Neural Machine Translation
Table 3: Improvements after using the dictionary of the development sets. The tokenized references of the English→Romanian and Chinese→English development sets have 26.7K and 46.6K running words respectively.
['#', 'Alignment', 'WMT En→Ro #entries', 'WMT En→Ro BLEU [%]', 'WMT En→Ro TER [%]', 'BOLT Zh→En #entries', 'BOLT Zh→En BLEU [%]', 'BOLT Zh→En TER [%]']
The English→Romanian dictionary covers 11.6% of the reference set, while the Chinese→English dictionary has 9.9% coverage. We observe larger improvement when using the dictionary entries in the alignment-assisted transformer system in comparison to the transformer baseline systems. Our system improves BLEU by 3.8%, while the baseline is improved only by 2.4% BLEU on the English→Romanian task. We also observe larger improvements in the Chinese→English case. This suggests that the maximum attention weights in alignment-assisted systems can point more accurately to the word being translated, allowing the use of more dictionary entries. This explains the larger improvements our systems achieve.