Adversarial Multi-task Learning for Text Classification
Table 2: Error rates of our models on 16 datasets against typical baselines. The numbers in brackets represent the improvements relative to the average performance (Avg.) of three single task baselines.
['[BOLD] Task', '[BOLD] Single Task LSTM', '[BOLD] Single Task BiLSTM', '[BOLD] Single Task sLSTM', '[BOLD] Single Task Avg.', '[BOLD] Multiple Tasks MT-DNN', '[BOLD] Multiple Tasks MT-CNN', '[BOLD] Multiple Tasks FS-MTL', '[BOLD] Multiple Tasks SP-MTL', '[BOLD] Multiple Tasks ASP-MTL']
The column of “Single Task” shows the results of vanilla LSTM, bidirectional LSTM (BiLSTM), stacked LSTM (sLSTM) and the average error rates of previous three models. The column of “Multiple Tasks” shows the results achieved by corresponding multi-task models. From this table, we can see that the performance of most tasks can be improved with a large margin with the help of multi-task learning, in which our model achieves the lowest error rates. More concretely, compared with SP-MTL, ASP-MTL achieves 4.1% average improvement surpassing SP-MTL with 1.0%, which indicates the importance of adversarial learning. It is noteworthy that for FS-MTL, the performances of some tasks are degraded, since this model puts all private and shared information into a unified space.