Generating Sentiment Lexicons for German Twitter
Table 2: Evaluation of dictionary-based approaches. HL – Hu:04, BG – Blair-Goldensohn:08, KH – Kim:04, ES – Esuli:06c, RR – Rao:09, AR – Awadallah:10
['[BOLD] Lexicon', '[BOLD] # of Terms', '[BOLD] Positive Expressions Precision', '[BOLD] Positive Expressions Recall', '[BOLD] Positive Expressions  [ITALIC] F1', '[BOLD] Negative Expressions Precision', '[BOLD] Negative Expressions Recall', '[BOLD] Negative Expressions  [ITALIC] F1', '[BOLD] Neutral Terms Precision', '[BOLD] Neutral Terms Recall', '[BOLD] Neutral Terms  [ITALIC] F1', '[BOLD] Macro  [ITALIC] F1', '[BOLD] Micro  [ITALIC] F1']
This time, the situation is much more varied, as different systems can achieve best results on just some aspects of certain classes but can hardly attain best overall scores in all categories. This is, for instance, the case for the positive and negative polarities, where the best precision scores are reached by the seed set in the first case and the label propagation algorithm of \newciteRao:09 in the second case. However, with respect to the recall, both of these polarity lists perform notably worse than the approach of \newciteEsuli:06c. Yet other systems—the matrix-vector method of \newciteBlair-Goldensohn:08 and the union of the three overall top-scoring systems respectively—reach the highest F1-scores for these two classes. Nevertheless, we can still notice three main tendencies in this evaluation: i) the method of \newciteEsuli:06c generally gets the highest recall of polar terms and, consequently, achieves the best precision in recognizing neutral words, but suffers from a low precision for the positive and negative polarities; ii) simultaneously five systems attain the same best F1-scores on recognizing neutral terms, which, in turn, leads to the best micro-averaged F1-results for all polarity classes; and, finally, iii) the system of \newciteBlair-Goldensohn:08 shows the best macro-averaged performance. This approach, however, is extremely susceptible to its hyper-parameter settings (in particular, we considered the maximum number of times the initial vector →v was multiplied with the adjacency matrix A as such a parameter and noticed a dramatic decrease of method ’s scores after the fifth iteration).