Multi-label Dataless Text Classification with Topic Modeling
Table 2: Performance comparison on the two datasets. The best and the second best results by dataless classifiers are highlighted in boldface and underlined respectively. #F1: Macro-F1 score; #AUC: Macro-AUC score.
['Method', 'Ohsumed # [ITALIC] F1', 'Ohsumed # [ITALIC] AUC', 'Delicious # [ITALIC] F1', 'Delicious # [ITALIC] AUC']
We observe that SMTM significantly outperforms all other dataless methods in terms of both Macro-F1 and Macro-AUC on both datasets. Among all dataless baselines in comparison, ESA delivers the best Macro-F1 scores on the two datasets, however with an expensive external knowledge base. We can also find that our approach is much better than DescLDA. Note that DescLDA is also built upon probabilistic topic models but designed for single-label classification. This suggests that our approach successfully discovers the underlying topical structure of multi-labeled documents, leading to better classification results. Thus, supervised classifiers should be preferred when training data is large in volume and of high-quality. Here, we are interested in conducting a deeper comparison of SMTM and supervised classifiers. Specifically, we will discuss a few scenarios in which our dataless classifier will be a more desired choice than supervised classifiers.