An Empirical Study of Adequate Vision Span for Attention-Based Neural Machine Translation
Table 1: Evaluation results on English-Japanese and German-English translation task. This table provides a comparison of the minimum window size of vision span the models can achieve with a modest loss of accuracy.
['Model', 'English-Japanese window (words)', 'English-Japanese BLEU(%)', 'English-Japanese RIBES', 'German-English window (words)', 'German-English BLEU(%)']
We evaluate the attention models to determine the minimum window size they can achieve with a modest loss of accuracy (0.5 development BLEU) compared to Flexible Attention with τ=∞. The scores of Global Attention (conventional attention model) and Local Attention Luong et al. For Local Attention, we found a window size of 21 (D=10) gives the best performance for En-Ja and De-En tasks. In this setting, Local Attention achieves an average window of 18.4 words in En-Ja task and 15.7 words in De-En task, as some sentences in the test corpus have fewer than 21 words.