Dynamically Composing Domain-Data Selection with Clean-Data Selection by “Co-Curricular Learning” for Neural Machine Translation
Table 8: Curriculum learning works better than retraining with a static, top selection, especially when the training dataset is small.
['[BOLD]  Model ', '[BOLD] Test Bleu  [BOLD] IWSLT15', '[BOLD] Test Bleu  [BOLD] WMT14']
What if we retrain on the static data, too? On Paracrawl, P11 vs. P9 are closer, but retraining on co-curriculum performs still better. In all cases, co-curricular learning gives the best results. We may tune the static selection for better results, but then it is the exact point of CL, to evolve the data re-weighting without the need of a hard cutoff on selection ratio.