Modeling Label Semantics for Predicting Emotional Reactions
Table 1: Comparison Results on ROCStories with Plutchik emotion labels
['[EMPTY]', '[BOLD] Model', '[BOLD] Precision', '[BOLD] Recall', '[BOLD] F1']
Among the baselines, the fine-tuned BERT base model obtains the best results. Adding label embeddings (section 3.1) to the basic BiLSTM via LEAM model provides substantial increase, more than 27 absolute points in F1. We swapped in BERT features instead of GloVe and found a further 3 point improvement. The BERT baseline beat both of these, but appending label sentences as additional input to fine-tuned BERT increased its performance by 1.4 F1 points. A further increase of 2 points in F1 is achieved by tracking label-label correlations through training loss and inference logits. In addition, adding semi-supervision yields the best gain of more than 4.9 points in F1 over basic BERT, providing a significant advance in state-of-the-art results for emotion inference in this dataset. This involved comparing the outputs of the Semi-supervision model with the above mentioned models after creating 100,000 random permutations. The Semi-supervision model achieved statistically significant improvement over all the baselines.