A Data-driven Neural Network Architecture for Sentiment Analysis
Table 4: Accuracy scores of NgramCNN variants and baselines
['[BOLD] Network', '[BOLD] Sent', '[BOLD] Imdb', '[BOLD] Phon', '[BOLD] Yelp']
We ran each trial on a 70/10/20 percent split for training, development and evaluation respectively. As we can see, NgramCNN models perform relatively poor on sentiment polarity dataset which is the smallest and contains the shortest documents. Next come the optimized Logistic Regression and Support Vector Machine models, together with SingleCNN network. The worst are the models based on LSTMs only (SingleLSTM and BLSTM-POOL). We got different results on movie reviews. Here we see that the first two NgramCNN models score over 91%. We also see that Fluctuating CNN is well behind, same as the two linear models and SingleCNN. All LSTM-based models achive very low results that are below 86%. Recurrent neural networks are obviously bad players on text analysis of long documents. Basic is the best of the three designs we proposed. Pyramid version that differs only in the intermediate subsampling layers performs similarly with slightly lower accuracy scores. Contrary, the Fluctuating architecture performed badly on every dataset. Apparently, it is not a good neural network design. We also saw that models based on recurrent neural networks perform well on short documents but considerably worse on longer ones. This is probably related with their internal design. Their memory is limited and they are not able to preserve long-term word relations. Moreover, those kinds of networks take longer to train. CNNs that are simpler in design are also faster, with training times comparable to those of Logistic Regression and Support Vector Machine. These later models scored comparably well on the small datasets where they can be a good choice.