Incremental Adaptation Strategies for Neural Network Language Models
Table 5: BLEU scores obtained by a baseline SMT (without and with an CSLM) and a project-adapted SMT with baseline (unadapted) CSLM and adapted CSLM. The first value in every cell is the BLEU score obtained with respect to the reference translation of the human translator; the second one is calculated with respect to all the 3 references created by the professional translators (i.e. obtained by post-edition) and an independent reference.
['Approach', 'Day 1', 'Day 2', 'Day 3', 'Day 4', 'Day 5']
In order to evaluate the impact of the CSLM adaptation on the SMT system, we performed various translation experiments. The BLEU scores of the various systems using the baseline and the adapted CSLMs are presented. We run tests with three different human translators - for the sake of clarity, we provide detailed results for one translator only. The observed tendencies are similar for the two other translators. First of all, one can see that the CSLM improves the BLEU score of the baseline systems between 2.3 to 3.4 BLEU points, e.g. for Day 2 from 44.07 to 46.61. Adapting the whole SMT system to the new data improves significantly the translation quality, e.g. from 46.61 to 52.01 for Day 2, without changing the CSLM. The proposed adaptation scheme of the CSLM achieves additional important improvements, in average 2.6 BLEU points. This gain is relatively constant for all days.