Group Sparse CNNs for Question Classification with Answer Sets
Table 2: Experimental results. Baselines: †sequential CNNs (α=β=0 in Eq. 5), ‡CNNs with global sparsity (β=0). WR: randomly initialized projection matrix. WQ: question-initialized projection matrix. WA: answer set-initialized projection matrix. There are three different classification settings for Yahoo: subcategory, top-level category, and top-level accuracies on unseen sub-labels.
['[EMPTY]', 'TREC', 'Insur.', 'DMV', 'Yahoo dataset sub', 'Yahoo dataset top', 'Yahoo dataset unseen']
The improvements are substantial for Insurance and DMV, but not as significant for Yahoo and TREC. One reason for this is the questions in Yahoo/TREC are shorter, which makes the group information harder to encode. Another reason is that each question in Yahoo/TREC has a single label, and thus can not fully benefit from group sparse properties. Besides the conventional classification tasks, we also test our proposed model on an unseen-label case. In these experiments, there are a few sub-category labels that are not included in the training data. However, we still hope that our model could still return the correct parent category for these unseen subcategories at test time. In the testing set of Yahoo dataset, we randomly add 100 questions whose subcategory labels are unseen in training set. The improvements are substantial due to the group information encoding.