Emotion Detection in Text: Focusing on Latent Representation
Table 5: Results of classification using two embedding models and bidirectional GRU. No meaningful differences was seen between the two models. Reported numbers are F1-measures.
['[BOLD] Emotion', '[BOLD] Numberbatch', '[BOLD] fastText']
Minimal pre-processing was done by converting text to lower case, removing the hashtags at the end of tweets and separating each punctuation from the connected token (e.g., awesome!! → awesome !!) and replacing comma and new-line characters with white space. The text, then, was tokenized using TensorFlow-Keras tokenizer. Top N terms were selected and added to our dictionary where N=100k for higher count emotions joy, sadness, anger, love and N=50k for thankfulness and fear and N=25k for surprise. Seven binary classifiers were trained for the seven emotions with a batch size of 250 and for 20 epochs with binary cross-entropy as the objective function and Adam optimizer. For training each classifier, a balanced dataset was created with selecting all tweets from the target set as class 1 and a random sample of the same size from other classes as class 0. For each classifier, 80% of the data was randomly selected as the training set, and 10% for the validation set, and 10% as the test set. As mentioned before we used the two embedding models, ConceptNet Numberbatch and fastText as the two more modern pre-trained word vector spaces to see how changing the embedding layer can affect the performance. It can be seen that the best performance was divided between the two embedding models with minor performance variations.