An Empirical Study of Adequate Vision Span for Attention-Based Neural Machine Translation
Table 2: Evaluation results with character-based English-Japanese NMT models
['Model', 'window', 'BLEU', 'RIBES']
Note that although the performance of character-based models can not compete with word-based model, the focus of this experiment is to examine the effects in terms of the reduction of the window size of vision span. For this dataset, the character-level tokenization will increase the length of input sequences by 6x on average. In this setting, the fine-tuned Flexible Attention model can achieve a reduction rate of 46% of the vision span. The results indicate that Flexible Attention can automatically adapt to the type of training data and learn to control the strength of penalty properly.