‘Who would have thought of that!’: A Hierarchical Topic Model for Extraction of Sarcasm-prevalent Topics and Sarcasm Detection
Table 7: Comparison of Various Approaches for Sarcasm Detection
['[BOLD] Approach', '[BOLD] P (%)', '[BOLD] R (%)', '[BOLD] F (%)']
We now use our sarcasm topic model to detect sarcasm, and compare it with two prior work. The task here is to classify a tweets as either sarcastic or not. We use the topic model for sarcasm detection using two methods: Log-likelihood based: The topic model is first learned using the training corpus where the distributions in the model are estimated. Then, the topic model performs sampling for a pre-determined number of samples, in three runs - once for each label. For each run, the log-likelihood of the tweet given the estimated distributions (in the training phase) and the sampled values of the latent variables (for this tweet) is computed. The label of the tweet is returned as the one with the highest log-likelihood. Sampling-based : Like in the previous case, the topic model first estimates distributions using the training corpus. Then, the topic model is learned again where the label l is assumed to be latent, in addition to the tweet-level latent variable z, and word-level latent variables s, and is. The value of l as learned by the sampler is returned as the predicted label. We compare our results with two previously existing techniques, \newcitebuschmeier2014impact and \newciteliebrecht2013perfect. We ensure that our implementations result in performance comparable to the reported papers. The two rely on designing sarcasm-level features, and training classifiers for these features. For these classifiers, the positive and negative labels are combined as non-sarcastic. As stated above, the test set is separate from the training set. Both prior work show poor F-score (around 18-19%) while our log-likelihood based approach achieves the best F-score of 41.34%. The low values, in general, may be because our corpus is large in size, and is diverse in terms of the topics. Also, features in \newciteliebrecht2013perfect are unigrams, bigrams and trigrams which may result in sparse features.