Context-Aware Monolingual Repair for Neural Machine Translation
Table 3: Results on contrastive test sets for specific contextual phenomena (deixis, lexical consistency, ellipsis (inflection), and VP ellipsis).
['[BOLD] model', '[BOLD] deixis', '[BOLD] lex. c.', '[BOLD] ell. infl.', '[BOLD] ell. VP']
For deixis, lexical cohesion and ellipsis (infl.) we see substantial improvements over both the baseline and CADec. The largest improvement over CADec (22.5 percentage points) is for lexical cohesion. However, there is a drop of almost 5 percentage points for VP ellipsis. We hypothesize that this is because it is hard to learn to correct inconsistencies in translations caused by VP ellipsis relying on monolingual data alone. There is no VP ellipsis in Russian, and when translating auxiliary “did” the model has to guess the main verb. When translating from Russian, main verbs are unlikely to be translated as the auxiliary “do” in English, and hence the VP ellipsis is rarely present on the English side. This implies the model trained using the round-trip translations will not be exposed to many VP ellipsis examples in training.