Jointly Learning to Align and Translate with Transformer Models
Table 3: Results on the alignment task (in Aer [%]). ‡Difference in Aer w.r.t. Giza++ (BPE-based) is statistically significant (p<0.001).
['Model', 'DeEn', 'RoEn', 'EnFr']
The layer average baseline provides relatively weak alignments, which are used for training our multi-task model. The improvement of the multi-task approach over the layer average baseline suggests that learning to translate helps produce better alignments as well. However still the multi-task approach falls short of the statistical and neural baselines, which have a strong advantage of having access to the full/partial target context. Exposing our model to the full target context gives the largest gains in terms of Aer. Note that full context results are directly comparable to \newciteZenkel2019 since both approaches do not leverage external knowledge from statistical models. We suspect that we are able to outperform \newciteZenkel2019 because we provide the full target context instead of only the to-be aligned target word. Finally, by supervising our model on the alignments obtained from Giza++ (Giza++ supervised) rather than layer average baseline, we outperform Giza++ and \newcitepeter2017:Foresight.