Large-Scale Multi-Label Text Classification on EU Legislation
Table 2: Results on eurlex57k for all, frequent, few-shot, zero-shot labels. Starred methods use the first 512 document tokens; all other methods use full documents. Unless otherwise stated, glove embeddings are used.
['[EMPTY]', 'All Labels  [ITALIC] RP@5', 'All Labels  [ITALIC] nDCG@5', 'All Labels Micro- [ITALIC] F1', 'Frequent  [ITALIC] RP@5', 'Frequent  [ITALIC] nDCG@5', 'Few  [ITALIC] RP@5', 'Few  [ITALIC] nDCG@5', 'Zero  [ITALIC] RP@5', 'Zero  [ITALIC] nDCG@5']
The naive baselines are weak, as expected. Interestingly, for all, frequent, and even few-shot labels, the generic bigru-att performs better than cnn-lwan, which was designed for lmtc. han also performs better than cnn-lwan for all and frequent labels. However, replacing the cnn encoder of cnn-lwan with a bigru (bigru-lwan) leads to the best results, indicating that the main weakness of cnn-lwan is its vanilla cnn encoder. Exact Match also performs better on zero-shot labels, for the same reason (i.e., the prior knowledge is intact). bigru-lwan, however, is still the best method in few-shot learning. Given that H+R contains enough information and is shorter than 500 tokens in 83% of our dataset’s documents, we also apply bert to the first 512 tokens of each document (truncated to bert ’s max. length), comparing to bigru-lwan also operating on the first 512 tokens. It fails, however, in zero-shot learning, since it does not have a component that exploits prior knowledge as is (i.e., all the components are fine-tuned on training data).