Memory-enhanced Decoder for Neural Machine Translation
Table 1: Case-insensitive BLEU scores on Chinese-English translation. Moses is the state-of-the-art phrase-based statistical machine translation system. For RNNsearch, we use the open source system Groundhog as our baseline. The strong baseline, denoted RNNsearch⋆, also adopts feedback attention and dropout. The coverage model on top of RNNsearch⋆ has significantly improved upon its published version (Tu et al., 2016), which achieves the best published result on this training set. For MemDec the number of cells is set to 8.
['SYSTEM', 'MT03', 'MT04', 'MT05', 'MT06', 'AVE.']
Clearly MemDec leads to remarkable improvement over Moses (+5.28 BLEU) and Groundhog (+4.78 BLEU). The feedback attention gains +1.06 BLEU score on top of Groundhog on average, while together with dropout adds another +0.83 BLEU score, which constitute the 1.89 BLEU gain of RNNsearch⋆ over Groundhog. Compared to RNNsearch⋆ MemDec is +2.89 BLEU score higher, showing the modeling power gained from the external memory. In this comparison MemDec wins with big margin (+1.46 BLEU score).