Detecting Sarcasm in Multimodal Social Platforms
Table 7: Silver Set evaluation using SVM fusion.
['[BOLD] Feature Set', '[BOLD] IG', '[BOLD] TU', '[BOLD] TW']
The top individual feature is n-gram ( 1- and 2-grams), roughly performing at close to 80% accuracy across all data sets. In fact, even though we use three disparate data sets, the performance figures for each feature are consistently the same as the the ranking of the features. This may suggest that users do not alter the way they use sarcasm across platforms, though the best way of testing this hypothesis would be to investigate whether models trained on one platform, e.g., Twitter, can approximate the performance found on the other platforms, e.g., Instagram, when models are trained on native data. Finally, merging several of the feature classes into one (combination) yields the best performance, exceeding 80% for all data sets. For all the three data sets, the model with n-grams + VFS outperformed the model solely trained on n-grams by a small margin. However, it was not better than using the combination features. When combining the visual features with the combination features, we achieve the highest performance in Instagram (82.3%) and Tumblr (81.0%). In Twitter, the fusion produces the second highest performance (80.0%) to the 80.5% yielded by combination features only. These results show that including simple, noisy image-related features can improve sarcasm detection, albeit by a small margin.