Online Learning for Effort Reduction in Interactive Neural Machine Translation
Table 2: Results of translation quality for all tasks in terms of TER [%] and BLEU [%]. We compare PB-SMT and NMT systems. Results that are statistically significantly better for each task and metric are boldfaced.
['[EMPTY]', '[EMPTY]', 'TER [%] PB-SMT', 'TER [%] NMT', 'BLEU [%] PB-SMT', 'BLEU [%] NMT']
For the sake of comparison, we first evaluated PB-SMT and NMT systems for performing classical translation post-editing. OL was also effective for the rest of tasks, obtaining consistent improvements, ranging from 3.0 to 4.7 TER points. The rest of OL systems also behaved good, achieving performances close to the systems trained on in-domain data. These results demonstrate that online learning is a good choice when developing translation systems with scarce data resources. In the XRCE and TED tasks, both TER and BLEU were significantly improved when fine-tuning the general system. We found improvements of approximately 4 TER points in each task. On the other hand, if we had available a large amount of in-domain data, the fine-tuning effectiveness is diluted. This is the case of the UFAL task: as this is a large in-domain corpus, to pre-train with Europarl had minor effects on the final NMT performance.