Mixture Models for Diverse Machine Translation: Tricks of the Trade
Table 2: Results on three WMT datasets: En-De, En-Fr, Zh-En have 10, 10, 3 human references respectively. We generate the same number of hypotheses as the number of references.
['[EMPTY]', 'Pairwise-BLEU en-de', 'Pairwise-BLEU en-fr', 'Pairwise-BLEU zh-en', 'BLEU en-de', 'BLEU en-fr', 'BLEU zh-en']
The BLEU of beam search is fairly close to human, implying a remarkably high generation quality. However, it severely lacks diversity, as Pairwise-BLEU is close to 80. This suggests a scenario similar to In contrast, sampling hypotheses are very diverse, typically even more so than human references on En-De and En-Fr, but have a very poor BLEU, as illustrated in Overall, hMup achieves the best trade-off between quality and diversity. Once again, mixture models offer a good trade-off between translation quality and diversity.