A Visual Attention Grounding Neural Modelfor Multimodal Machine Translation
Table 5: Ablation analysis on visual-text attention mechanism in the Multi30K German dataset.
['Method', 'English → German BLEU', 'English → German METEOR']
As can be seen, the performance of the changed translation model is obviously worse than the full VAG-NMT in both experiments. This observation suggests that the visual-attention mechanism is critical in improving the translation performance. The model improvement comes from the attention mechanism influencing the model’s objective function and decoder’s initialization.