Word Vector Enrichment of Low Frequency Words in the Bag-of-Words Model for Short Text Multi-class Classification Problems
Table 4: DRC Dataset (2,159 instances, 64 Classes): Domain Specific Word Vector.
['[EMPTY]', '[ITALIC] n', '[ITALIC] k', '[BOLD] Micro Recall Baseline', '[BOLD] Micro Recall WV Enrichment', '[BOLD] Micro Recall Error Reduction', '[BOLD] Macro Recall Baseline', '[BOLD] Macro Recall WV Enrichment', '[BOLD] Macro Recall Error Reduction']
The baseline results for the SVM classifier again shows much higher performance than the baseline MNB result. When we enrich the bag-of-words model, we get a gain in both micro and macro recall for the MNB classifier. For the SVM classifier we get a much smaller gain in micro recall, however, the results are statistically significant (p<0.0005). All of these results are statistically significant (p<0.00005). However, despite the improvement with the enrichment process, the linear classifiers still perform very poorly on this dataset. An alternative treatment of this dataset is to configure the classification algorithms to produce a ranked top-3 predictions of the class label which a human “coder” can then select from (cf.