What does Attention in Neural Machine TranslationPay Attention to?
Table 5: Average loss between attention generated by input-feeding and non-recurrent systems and the manual alignment over RWTH German-English data.
['[EMPTY]', 'non-recurrent', 'input-feeding']
Here the losses between the attention produced by each system and the human alignments is reported. As expected, the difference in attention losses are in line with AER.