Deep Short Text Classification with Knowledge Powered Attention
Table 4: The impact of different embedding tunning methods on our model
['Model', 'Weibo', 'Topic', 'Product Review', 'News Title']
Besides, STCKA-rand performs more poorly than STCKA-static on small training datasets such as Weibo and Topic. The reason could be twofold: (1) The amount of labeled samples in the two experimental datasets is too small to tune reliable embeddings from scratch for the in-vocabulary words (i.e., existing in the training data); (2) A lot of out-of-vocabulary words, i.e., absent from the training data, but exist in the testing data. However, STCKA-rand outperforms STCKA-static on large-scale training data such as News Title. Because large-scale training data alleviates two above-mentioned reasons and enables STCKA-rand to learn task-oriented embeddings which are better for different classification tasks.