A Deep Reinforced Sequence-to-Set Model for Multi-Label Text Classification
Table 5: Performance on rebuilt uncorrelated RCV1-V2 test set. Detailed explanations of symbols can be found in Table 3.
['[BOLD] Models', '[BOLD] HL(-)', '[BOLD] P(+)', '[BOLD] R(+)', '[BOLD] F1(+)']
First, we build a collection of uncorrelated labels. The maximum correlation coefficient between these labels is 0.28, indicating that these labels are uncorrelated. Then, we pick out the samples whose labels are all from the uncorrelated label set. We use these samples to rebuild a label-uncorrelated dataset. The reason is that the Seq2Seq model makes a strict requirement on the label order, while there is not a strict order between labels in this case, leading that any label permutation will be unreasonable. The Seq2Seq model will suffer from serious wrong penalty. However, both of our proposed methods outperform the Seq2Seq model and the BR algorithm by a large margin, which shows that our methods are capable of achieving good performance even in the case where there is not a strict order between labels. Our models are trained using reinforcement learning, and the calculation of reward r satisfies swapping-invariance, leading that they reduce the dependence on the label order. This is the main reason for the robustness and universality of our models.