Learning Concept Embeddings for Dataless Classification via Efficient Bag of Concepts Densification
Table 7: Evaluation results of dataless document classification of fine-grained classes measured in micro-averaged F1 along with # of dimensions (concepts) in the BoC at which corresponding performance is achieved.
['Method ESA', 'Hockey x Baseball 94.60', 'Hockey x Baseball @425', 'Autos x Motorcycles 72.70', 'Autos x Motorcycles @325', 'Guns x Mideast x Misc 70.00', 'Guns x Mideast x Misc @500']
As we can notice, ESA achieves its peak performance with a few hundred dimensions of the sparse BoC vector. Using our densification mechanism, both the CRX & CCX models achieve equal performance to ESA at many fewer dimensions. Densification using the CRX model embeddings gives the best F1 scores on the three tasks. Interestingly, the CRX model improves the F1 score by ∼7% using only 14 concepts on Autos vs. Motorcycles, and by ∼3% using 70 concepts on Guns vs. Mideast vs. Misc. The CCX model, still performs better than ESA on 2 out of the 3 tasks. Both WEmax and WEhung improve the performance over ESA but not as our CRX model.