WASSA-2017 Shared Task on Emotion Intensity
Table 5: Pearson correlations (r) of emotion intensity predictions with gold scores. Best results for each column are shown in bold: highest score by a feature set, highest score using a single lexicon, and highest score using feature set combinations.
['[BOLD] anger', '[BOLD] Pearson correlation r  [BOLD] anger', '[BOLD] Pearson correlation r  [BOLD] fear', '[BOLD] Pearson correlation r  [BOLD] joy', '[BOLD] Pearson correlation r  [BOLD] sad.', '[BOLD] Pearson correlation r  [BOLD] avg.']
We developed the baseline system by learning models from each of the Tweet Emotion Intensity Dataset training sets and applying them to the corresponding development sets. Once the system parameters were frozen, the system learned new models from the combined training and development corpora. This model was applied to the test sets. The last column ‘avg.’ shows the macro-average of the correlations for all of the emotions. Among the lexicons, NRC-Hash-Emo is the most predictive single lexicon. Lexicons that include Twitter-specific entries, lexicons that include intensity scores, and lexicons that label emotions and not just sentiment, tend to be more predictive on this task–dataset combination. NRC-Aff-Int has real-valued fine-grained word–emotion association scores for all the words in NRC-EmoLex that were marked as being associated with anger, fear, joy, and sadness. The correlations for anger, fear, and joy are similar (around 0.65), but the correlation for sadness is markedly higher (0.71). SentiStrength focuses solely on positive and negative classes, but provides numeric scores for each. To assess performance in the moderate-to-high range of the intensity scale, we calculated correlation scores over a subset of the test data formed by taking only those instances with gold emotion intensity scores ≥0.5. We observe that the correlation scores are in general lower here in the 0.5 to 1 range of intensity scores than in the experiments over the full intensity range. This is simply because this is a harder task as now the systems do not benefit by making coarse distinctions over whether a tweet is in the lower range or in the higher range.