Toward Making the Most of Context in Neural Machine Translation
Table 5: Accuracy (\%) of discourse phenomena. * different data and system conditions, only for reference.
['[BOLD] Model', 'deixis', 'lex.c.', 'ell.infl.', 'ell.VP']
We also want to examine whether the proposed model actually learns to utilize document context to resolve discourse inconsistencies that context-agnostic models cannot handle. We use contrastive test sets for the evaluation of discourse phenomena for English-Russian by \citeauthorvoita2018context [\citeyearvoita2018context]. There are four test sets in the suite regarding deixis, lexicon consistency, ellipsis (inflection), and ellipsis (verb phrase). Each testset contains groups of contrastive examples consisting of a positive translation with correct discourse phenomenon and negative translations with incorrect phenomena. The goal is to figure out if a model is more likely to generate a correct translation compared to the incorrect variation. Our model is better at resolving discourse consistencies compared to context-agnostic baseline. \citeauthorvoita2018context [\citeyearvoita2018context] use a context-agnostic baseline, trained on 4\times larger data, to generate first-pass drafts, and perform post-processings, which is not directly comparable, but would be easily incorporated with our model to achieve better results.