Semantic-Unit-Based Dilated Convolution for Multi-Label Text Classification
Table 4: Performance of the models with different attention mechanisms on the RCV1-V2 test set. HL, P, R, and F1 denote hamming loss, micro-precision, micro-recall and micro-F1, respectively (p<0.05).
['[BOLD] Models', '[BOLD] HL(-)', '[BOLD] P(+)', '[BOLD] R(+)', '[BOLD] F1(+)']
Except for the insignificant effect of the conventional attention mechanism mentioned above, it can be found that the high-level representations generated by the MDC contribute much to the performance of the Seq2Seq model for multi-label text classification, which improves about 0.9 micro-F1 score. Moreover, simple additive attention mechanism, which is equivalent to the element-wise addition of the representations of MDC and those of the conventional mechanism, achieves similar performance to the single MDC, which also demonstrates that conventional attention mechanism in this task makes little contribution. As to our proposed hybrid attention, which is a relatively complex combination of the two mechanisms, can improve the performance of MDC. This shows that although conventional attention mechanism for word-level information does not influence the performance of the SeqSeq model significantly, the hybrid attention which extracts word-level information based on the generated high-level semantic information can provide some information about important details that are relevant to the most contributing semantic units.