Distant supervision for emotion detection using Facebook reactions
Table 2: Results on the development set (Affective development). avg f is the micro-averaged f-score.
['[BOLD] Feature', '[BOLD] anger prec,rec,f', '[BOLD] joy prec,rec,f', '[BOLD] sadness prec,rec,f', '[BOLD] surprise prec,rec,f', 'avg f']
In selecting appropriate features, we mainly relied on previous work and intuition. We experimented with different combinations, and all tests were still done on Affective development, using the pages for the best model (B-M) described above as training data. Future work will further explore the simultaneous selection of features and page combinations. First, a simple tf-idf bag-of-word mode works already very well, to the point that the other textual and lexicon-based features don’t seem to contribute to the overall f-score (0.368), although there is a rather substantial variation of scores per class. Second, Google embeddings perform a lot better than Facebook embeddings, and this is likely due to the size of the corpus used for training. Retrofitting doesn’t seem to help at all for the Google embeddings, but it does boost the Facebook embeddings, leading to think that with little data, more accurate task-related information is helping, but corpus size matters most. Third, in combination with embeddings, all features work better than just using tf-idf, but removing the Lexicon feature, which is the only one based on hand-crafted resources, yields even better results. Then our best model (B-M) on development data relies entirely on automatically obtained information, both in terms of training data as well as features.