Deep Short Text Classification with Knowledge Powered Attention
Table 2: Accuracy of compared models on different datasets.
['Model', 'Weibo', 'Topic', 'Product Review', 'News Title']
Our model outperforms traditional Deep Neural Networks (DNNs), including CNN, RCNN, CharCNN, BiLSTM-MP and BiLSTM-SA, without using any knowledge. The main reason is that our model enriches the information of short texts with the help of KBs. Specifically, we incorporate prior knowledge in KBs into DNNs as explicit features which have a great contribution to short text classification. Compared with traditional DNNs, our model is more like a human being who has intrinsic ability to make decisions based on observation (i.e., training data for machines) as well as existing knowledge. In addition, our model also performs better than KPCNN since our model is able to pay more attention to important knowledge due to the attention mechanism. We use C-ST and C-CS attention to measure the importance of knowledge from two aspects and adaptively assign a proper weight to each knowledge of different short texts.