Do all Roads Lead to Rome?Understanding the Role of Initialization in Iterative Back-Translation
Table 2: Average similarity across initial systems and final systems, as well as each initial system and its corresponding final system (BLEU).
['[EMPTY]', 'DE-EN init', 'DE-EN init', 'DE-EN final', 'EN-DE init', 'EN-DE init', 'EN-DE final']
As it can be seen, final systems trained through iterative back-translation tend to produce substantially more similar translations than the initial systems used in their warmup (49.3 vs 28.2 for German-to-English and 42.9 vs 24.3 for English-to-German). This suggests that iterative back-translation does not only converge to solutions of similar quality, but also to solutions that have a similar behavior. Interestingly, this also applies to systems that follow a fundamentally different paradigm as it is the case of RBMT. In relation to that, note that the similarity of each final system and its corresponding initial system is rather low, which reinforces the idea that the solution found by iterative back-translation is not heavily dependent on the initial system.