Modeling Voting for System Combination in Machine Translation
Table 2: Results on the Chinese-English task. The evaluation metric is case-insensitive tokenized BLEU. “All” is the concatenation of all test sets. The translations of the top three single MT systems are the inputs of the bottom three system combination methods. “††”: significantly better than “Transformer-L2R” (p<0.01). “‡” and “‡‡”: significantly better than “Jane” (p<0.05 and p<0.01). “**”: significantly better than “Hier” (p<0.01).
['Method', 'NIST02', 'NIST03', 'NIST04', 'NIST05', 'NIST08', 'All']
We find that our method outperforms the best single system Trans-L2R, the statistical combination method Jane, and the neural combination method Hier. All the differences are statistically significant (p<0.01). The superiority over Jane and Hier suggests that combining the merits of analyzing the dependencies between hypotheses and end-to-end training of neural networks helps to generate better translations.