Norm-Based Curriculum Learning for Neural Machine Translation
Table 3: Effects of different λm of the norm-based model competence function and λw of the norm-based sentence weight function.
['[ITALIC] λm 1.0', 'Dev. 26.63', '[ITALIC] λw 0', 'Dev. 26.71']
For each experiment, we kept the other parameters unchanged and only adjusted the hyperparameter. For λm, controlling curriculum length, the higher the value, the longer the curriculum length. When setting λm to 2.5 with the curriculum length of nearly 29K steps, it achieves the best performance. For λw, the scaling sentence weight of the objective function, one achieves satisfactory results with a value of 0.5, which maintains the right balance between the learning of simple and hard examples.