Data Diversification: An Elegant Strategy For Neural Machine Translation
Table 11: Improvements of data diversification under conditions with- and without- dropout in the IWSLT’14 English-German and German-English.
['[BOLD] Task', '[BOLD] Baseline', '[BOLD] Ours', '[BOLD] Gain']
First, given that parameter initialization affects diversity, it is logical to assume that dropout will magnify the diversification effects. However, our empirical results did not support this. We ran experiments to test whether non-zero dropout magnify the improvements of our method over the baseline. We trained the single-model baseline and our data diversification’s final model in cases of dropout=0.3 and dropout=0, in the IWSLT’14 English-German and German-English. We used factor k=1 in these experiments. However, the gains made by our data diversification with dropout are not particularly higher than the non-dropout counterpart. This suggests that dropout may not contribute to the diversity of the synthetic data.