Learning Context-Sensitive Convolutional Filters for Text Processing
Table 2: Test error rates on document classification tasks (in percentages). S-model indicates that the model has one single convolutional filter, while M-model indicates that the model has multiple convolutional filters. Results marked with ∗ are reported by Zhang et al. (2015), † are reported by Conneau et al. (2016), and ‡ are reported by Lin et al. (2017).
['[BOLD] Model', '[BOLD] Yelp P.', '[BOLD] DBpedia']
To explicitly explore whether our ACNN model can leverage the input-aware filter weights for better sentence representation, we perform a comparison between the basic CNN and ACNN models with only a single filter, which are denoted as S-CNN, S-ACNN, respectively (this setting may not yield best overall performance, since only a single filter is used, but it allows us to isolate the impact of adaptivity). As a result, with only one convolutional filter and thus very limited modeling capacity, our S-ACNN model tends to be much more expressive than the basic CNN model, due to the flexibility of applying different filters to different sentences. We further experiment on both ACNN and CNN models with multiple filters. Although we only use one convolution layer for our ACNN model, it already outperforms other CNN baseline methods with much deeper architectures. Moreover, our method exhibits higher accuracy than n-grams, which is a very strong baseline as shown in Zhang et al. We attribute the superior performance of the ACNN framework to its stronger (adaptive) feature-extraction ability. Moreover, our M-ACNN also achieves slightly better performance than self-attentive sentence embeddings proposed in Lin et al.