A Joint Many-Task Model:Growing a Neural Network for Multiple NLP Tasks
Table 10: Effects of the order of training.
['[EMPTY]', 'JMTall', 'Random']
S6SS2SSS0Px6 Order of training Our hypothesis is that it is important to start from the lower-level tasks and gradually move to the higher-level tasks. We see that the scores of the semantic tasks drop by the random strategy. In our preliminary experiments, we have found that constructing the mini-batch samples from different tasks also hampers the effectiveness of our model, which also supports our hypothesis.