Learning to Remember Translation History with a Continuous Cache
Table 6: Comparison of shallow fusion (i.e., words as cache values) and deep fusion (i.e., continuous vectors as cache values) in the News domain.
['[BOLD] Model', '[BOLD] Tune', '[BOLD] Test']
Different from \newciteGu:2017: arXiv, the shallow fusion does not achieve improvement over the baseline system. One possible reason is that the generated words are less repetitive with those in the translation history than those similar sentences retrieved from the training corpus. Accordingly, storing words in the cache encourages lexical consistency at the cost of introducing noises, while storing continuous vectors is able to improve this problem by doing fusion in a soft way. In addition, the continuous vectors can store useful information beyond a single word, which we will show later.