Generalizing Back-Translation in Neural Machine Translation
Table 2: IBM-1 model entropy and perplexity (Ppl) on the training and development set for the controlled scenario using different synthetic generation methods.
['[EMPTY]', 'Entropy En → De', 'Ppl Train', 'Ppl test2015']
These are the perplexities (Ppl) of the model on the training and development data and the entropy of a target-to-source IBM-1 model Brown et al. The training set Ppl varies strongly with each generation method since each produces hypotheses of differing quality. All methods with a restricted search space have a larger translation entropy and smaller training Ppl than natural data. This is due to the sentences being less noisy and also the translation options being less varied. Unrestricted sampling seems to overshoot the statistics of natural data, attaining higher entropy values.