Interactive Attention for Neural Machine Translation
Table 2: BLEU-4 scores (%) of the conventional attention-based model (RNNsearch⋆-80), the neural network based coverage model (NN-Cover-80) [Tu et al.2016] and our Interactive Attention model (NMTIA-80). “-80” means the models are trained with the sentences of length up to 80 words, which is consistent with the setting in [Tu et al.2016]. The “*” and “+” denote that the results are significantly (p<0.01) better than those of RNNsearch⋆-80 and NN-Cover-80 respectively.
['Systems', 'MT03', 'MT04', 'MT05', 'MT06', 'Average']
\newcite Tu2016 proposed two coverage models to let the NMT system to consider more about untranslated source words. Basically, they maintain a coverage vector for each hidden state for source to keep track of the attention history and feed the coverage vector to the attention model to help adjust future attention. Although we do not maintain a coverage vector, our Interactive Attention can potentially do similar things, therefore subsuming coverage models as special cases. Since the neural network based coverage model generally yields better results, we mainly compare with the neural network based coverage model.