Deep Short Text Classification with Knowledge Powered Attention
Table 3: The setting of hyper-parameter λ on our model
['Model', 'Weibo', 'Topic', 'Product Review', 'News Title']
The goal of this part is to verify the effectiveness of two attention mechanisms (i.e., C-ST and C-CS attention). We manually tune the hyper-parameter γ to explore the relative importance of C-ST and C-CS attention. In general, the model with γ=0.25 works better, but the advantage is not always there for different datasets. For instance, the model with γ=0.50 performs the best on Topic dataset. When γ is equal to 0 or 1, the model performs poorly on all four datasets. Using C-ST attention only (γ=1.00), the model neglects the relative importance of each concept, which leads to poor performance. On the other hand, merely using C-CS attention (γ=0.00), the model ignores the semantic similarity between the short text and concepts. In this case, an improper concept may be assigned with a larger weight which also results in poor performance.