Multi-Agent Cross-Translated Diversification for Unsupervised Machine Translation
Table 3: BLEU comparison of MACD vs. no cross-translation variants in the WMT’14 English-French (En-Fr), WMT’16 English-German (En-De) and English-Romanian (En-Ro) tasks.
['[BOLD] Method', '[BOLD] En-Fr', '[BOLD] Fr-En', '[BOLD] En-De', '[BOLD] De-En', '[BOLD] En-Ro', '[BOLD] Ro-En']
In particular, the MAD (1st level, n=1) variant fails to improve as the final supervised model is trained on the same synthetic data that the UMT agent is already trained on. The variant MAD (1st level, n=2) is in fact inspired from nguyen2019data, which improves supervised and semi-supervised MT. However, it fails to impress in the unsupervised setup, possibly due to the lack of supervised agents. The variant MAD (2nd level, n=2) also fails because the 2nd level synthetic data is already optimized during back-translation training of the UMT agents, leaving the supervised model with no extra information to exploit. On the other hand, cross-translation enables MACD to translate the second-level data by an agent other than the first-level agent. In this strategy, the second agent produces targets that the first agent is not aware of, while the second agent receives as input the sources that are foreign to it. This process creates corrupted but new information, which the final supervised MT model can take advantage of to improve the overall MT performance.