Leveraging Cognitive Features for Sentiment Analysis
Table 2: Results for different feature combinations. (P,R,F)→ Precision, Recall, F-score. Feature labels Uni→Unigram features, Sn→Sentiment features, Sr→Sarcasm features and Gz→Gaze features along with features related to reading difficulty
['[BOLD] Classifier', '[BOLD] Näive Bayes  [BOLD] Dataset 1', '[BOLD] Näive Bayes  [BOLD] Dataset 1', '[BOLD] Näive Bayes  [BOLD] Dataset 1', '[BOLD] SVM  [BOLD] Dataset 1', '[BOLD] SVM  [BOLD] Dataset 1', '[BOLD] SVM  [BOLD] Dataset 1', '[BOLD] Multi-layer NN  [BOLD] Dataset 1', '[BOLD] Multi-layer NN  [BOLD] Dataset 1', '[BOLD] Multi-layer NN  [BOLD] Dataset 1']
We observe the maximum accuracy with the complete feature-set comprising Sentiment, Sarcasm and Thwarting, and Cognitive features derived from gaze data. For this combination, SVM outperforms the other classifiers. The novelty of our feature design lies in (a) First augmenting sarcasm and thwarting based features (Sr) with sentiment features (Sn), which shoots up the accuracy by 3.1% for Dataset1 and 7.8% for Dataset2 (b) Augmenting gaze features with Sn+Sr, which further increases the accuracy by 0.6% and 1.5% for Dataset 1 and 2 respectively, amounting to an overall improvement of 3.7% and 9.3% respectively. It may be noted that the addition of gaze features may seem to bring meager improvements in the classification accuracy but the improvements are consistent across datasets and several classifiers. Still, we speculate that aggregating various eye-tracking parameters to extract the cognitive features may have caused loss of information, there by limiting the improvements. For example, the graph based features are computed for each participant and eventually averaged to get the graph features for a sentence, thereby not leveraging the power of individual eye-movement patterns. We intend to address this issue in future.