Generating Sentiment Lexicons for German Twitter
Table 3: Evaluation of corpus-based approaches. TKM – Takamura:05, VEL – Velikovich:10, KIR – Kiritchenko:14, SEV – Severyn:15
['[BOLD] Lexicon', '[BOLD] # of Terms', '[BOLD] Positive Expressions Precision', '[BOLD] Positive Expressions Recall', '[BOLD] Positive Expressions  [ITALIC] F1', '[BOLD] Negative Expressions Precision', '[BOLD] Negative Expressions Recall', '[BOLD] Negative Expressions  [ITALIC] F1', '[BOLD] Neutral Terms Precision', '[BOLD] Neutral Terms Recall', '[BOLD] Neutral Terms  [ITALIC] F1', '[BOLD] Macro  [ITALIC] F1', '[BOLD] Micro  [ITALIC] F1']
This time, we can observe a clear superiority of Takamura et al.’s method, which not only achieves the best recall and F1 in recognizing positive and negative items but also attains the highest micro- and macro-averaged results for all three polarity classes. The cardinality of the other induced lexicons, however, is much smaller than the size of Takamura et al.’s polarity list. Moreover, these lexicons also show absolutely identical scores for the negative expressions as the original seed set. Since these results were somewhat unexpected, we decided to investigate the reasons for possible problems. As it turned out, the macro-averaged F1-values of these methods were rapidly going down on the held-out development set as the number of their induced polar terms increased. Since we considered the lexicon size as one of the hyper-parameters of the tested approaches, we immediately stopped populating these lexicons when we noticed a decrease in their results. As a consequence, only the highest-ranked terms (all of which had the positive polarity) were included in the final lists.