Dynamic Multi-Level Multi-Task Learning for Sentence Simplification
Table 3: Results on dynamic vs. static mixing ratio (FKGL: lower is better).
['Models', 'BLEU', 'FKGL', 'SARI']
We view the problem of learning the right mixing of tasks as a sequential control problem, where the controller’s goal is to decide the next task/action after every ns training steps in each task-sampling round tb. We model the controller as a M-armed bandits, where it selects a sequence of actions/arms over the current training trajectory to maximize the expected future payoffs (see Fig. At each round tb, the controller selects an arm based on noisy value estimates and observes rewards rtb for the selected arm (we use the negative validation loss of the primary task as the reward in our setup). One problem in bandits learning is the trade-off between exploration and exploitation, where the agent needs to make a decision between taking the action that yields the best payoff on current estimates, or explore new actions whose payoffs are not yet certain. Let πtb be the policy of the bandit controller at round tb, we define this to be: πtb(ai)=exp(Qtb,i/τ)/M∑j=1exp(Qtb,j/τ) (3) where Qtb,i is the estimated action value of each arm i at round tb, and τ is the temperature. If Q0,i is the initial value estimate of arm i, then Qtb,i is the exponentially weighted mean with the decay rate α: Qtb,i=(1−α)tbQ0,i+tb∑k=1α(1−α)tb−krk (4) To further help the exploration process, we follow the principle of optimism under uncertainty and set Q0,i to be above the maximum empirical rewards. Also, we further show ablation study in Sec.