Adaptive Scheduling for Multi-Task Learning
Table 1: Comparison of scheduling methods, measured by BLEU scores. Best results in bold.
['Method', 'Task 1 (En-De) Dev', 'Task 1 (En-De) Test', 'Task 2 (En-Fr) Dev', 'Task 2 (En-Fr) Test']
Considering the amount of training data, we trained single task baselines for 400K and 600K steps for En-De and En-Fr respectively, where multi-task models are trained for 900K steps after training. All reported scores are the average of the last 20 checkpoints. Within each general schedule type, model selection was performed by maximizing the average development BLEU score between the two tasks.