Mixture Models for Diverse Machine Translation: Tricks of the Trade
Table 1: Results on the WMT’17 En-De dataset with K=3 for mixture models with no dropout (no), dropout=0.1 in both the E-step and M-step (E&M), or in the M-step only (M). For all model variants, dropout in the E-step causes the model to ignore the latent variable.
['[EMPTY]', 'Pairwise-BLEU no', 'Pairwise-BLEU E&M', 'Pairwise-BLEU M', 'BLEU no', 'BLEU E&M', 'BLEU M']
Unfortunately, entirely disabling dropout exacerbates overfitting and gives quite worse translation quality, i.e., lower BLEU ( This allows the model to make consistent use of the experts and helps them to diversify ( We adopt this dropout scheme for subsequent experiments. In this section, we further investigate the impact of dropout on the responsibility computation, using hMup as an example.