Structured Attention Networks
Table 3: Translation performance as measured by BLEU (higher is better) on character-to-word and word-to-word Japanese-English translation for the three different models.
['[EMPTY]', 'Simple', 'Sigmoid', 'Structured']
Sigmoid attention outperforms simple (softmax) attention on the character-to-word task, potentially because it is able to learn many-to-one alignments. On the word-to-word task, the opposite is true, with simple attention outperforming sigmoid attention. Structured attention outperforms both models on both tasks, although improvements on the word-to-word task are modest and unlikely to be statistically significant.