Investigating Capsule Networks with Dynamic Routing for Text Classification
Table 2: Comparisons of our capsule networks and baselines on six text classification benchmarks.
['[EMPTY]', '[BOLD] MR', '[BOLD] SST2', '[BOLD] Subj', '[BOLD] TREC', '[BOLD] CR', '[BOLD] AG’s']
In our experiments, the evaluation metric is classification accuracy. From the results, we observe that the capsule networks achieve best results on 4 out of 6 benchmarks, which verifies the effectiveness of the capsule networks. In particular, our model substantially and consistently outperforms the simple deep neural networks such as LSTM, Bi-LSTM and CNN-rand by a noticeable margin on all the experimental datasets. Capsule network also achieves competitive results against the more sophisticated deep learning models such as LR-LSTM, Tree-LSTM, VC-CNN and CL-CNN. Note that Capsule-B consistently performs better than Capsule-A since Capsule-B allows to learn more meaningful and comprehensive text representation. For example, a combination of N-gram convolutional layer with filter windows of {3,4,5} can capture the 3/4/5-gram features of the text which play a crucial role in text modeling. Multi-label text classification is, however, a more challenging practical problem. From single-label to multi-label (with n category labels) text classification, the label space is expanded from n to 2n, thus more training is required to cover the whole label space. For single-label texts, it is practically easy to collect and annotate the samples. However, the burden of collection and annotation for a large scale multi-label text dataset is generally extremely high. How deep neural networks (e.g., CNN and LSTM) best cope with multi-label text classification still remains a problem since obtaining large scale of multi-label dataset is a time-consuming and expensive process. In this section, we investigate the capability of capsule network on multi-label text classification by using only the single-label samples as training data. With feature property as part of the information extracted by capsules, we may generalize the model better to multi-label text classification without an over extensive amount of labeled data.