Reinforcement Learning based Curriculum Optimization forNeural Machine Translation
Table 1: BLEU scores on Paracrawl and WMT En-Fr datasets with uniform, heuristic and learned curricula.
['[EMPTY]', '[BOLD] Paracrawl', '[BOLD] WMT']
Uniform baselines consist of: Uniform – standard NMT training; Uniform (6-bins) – uniform sampling over all bins; and Uniform (bookends) – uniform sampling over just the best and worst bins. Surprisingly, 6-bins performs better than the standard NMT baseline. We hypothesize that this can be attributed to more homogeneous mini-batches. Our exploration strategy for Q-learning (see This mimics the gradual annealing of the online curriculum, so one possibility is that the agent is simply choosing the cleanest bin whenever it can, and its good performance comes from the enforced period of exploration.