Combining SMT and NMT Back-Translated Data for Efficient NMT
Table 1: Results for experimental procedure validation: checking that it is reasonable to use constant number of epochs, not constant amount of training effort, in the experiments.
['[EMPTY]', '1M  [ITALIC] base.- 13 Epochs', '1M  [ITALIC] base.- 39 Epochs-', '1M  [ITALIC] base + 2M  [ITALIC] NMTsynth']
It can be observed that increasing the number of epochs from 13 to 39 when using just the 1M base training set does not increase the performance over using just 13 epochs (i.e. not compensating the relatively smaller training set with more epochs), rather it deteriorates it. From these results we conclude that there is no reason to believe that the positive effects of using back-translation is caused by an effectively larger training effort, rather than by the advantage of the larger training set itself. We therefore also conclude that it is reasonable to keep the number of epochs constant across experiments, rather than fixing the amount of training effort as measured by steps/mini-batches, and we do the former throughout the rest of the paper.