Dynamically Composing Domain-Data Selection with Clean-Data Selection by “Co-Curricular Learning” for Neural Machine Translation
Table 5: EM-style optimization further improves domain curriculum. But, overall, it has a small impact on deep models.
['[BOLD] Curriculum', '[BOLD] Test Bleu  [BOLD] IWSLT15', '[BOLD] Test Bleu  [BOLD] WMT14']
Deep models. 1024 dimensions x 8 layers) trained on the final co-curriculum. P8 performs slightly better than the non-bootstrapped version P7 on Paracrawl: +0.6 BLEU on WMT14 test and +0.2 on IWSLT15 test. The differences on the WMT data appear to be smaller (W8 vs. W7). So, curriculum bootstrapping has a small impact overall on deep models.