Facebook Reaction-Based Emotion Classifier as Cue for Sarcasm Detection
Table 4. Performance comparison against other methods for different data sets and varying levels of annotator agreement.
['Set', 'Method', '[BOLD] Agree 1  [ITALIC] Accuracy', '[BOLD] Agree 1  [ITALIC] F1', '[BOLD] Agree 1  [ITALIC] Recall', '[BOLD] Agree 1  [ITALIC] Precision', '[BOLD] Agree 2  [ITALIC] Accuracy', '[BOLD] Agree 2  [ITALIC] F1', '[BOLD] Agree 2  [ITALIC] Recall', '[BOLD] Agree 2  [ITALIC] Precision', '[BOLD] Agree 3  [ITALIC] Accuracy', '[BOLD] Agree 3  [ITALIC] F1', '[BOLD] Agree 3  [ITALIC] Recall', '[BOLD] Agree 3  [ITALIC] Precision']
PBLGA method on the other hand doesn’t perform well in terms of F1 score but improves in accuracy with the level of agreement which appeared surprising. After taking a look at the classification output of PBLGA it was found that opposite from the content based methods, PBLGA is very selective on where to label sarcasm, therefore the fewer cases of sarcasm present in the ground truth, the better the accuracy since it will classify most of the non-sarcasm correctly, nevertheless it suffers alot in recall. The Emo-Based sarcasm classifier receives emotions from a pattern based approach which can provide more context. Additionally the candidate filtering process as well as the Distance Ratio and Score Ratio measurements introduced in the methodology make the classifier not so generous yet at the same time not overly selective when labeling a text as sarcasm.