SemEval-2016 Task 4: Sentiment Analysis in Twitter
Table 9: Historical results for Subtask A “Message Polarity Classification”. The systems are ordered by their score on the Tweet 2016 dataset; the rankings on the individual datasets are indicated with a subscript. The meaning of “(*)” is as in Table 8.
['[BOLD] #', '[BOLD] System', '[BOLD] 2013  [BOLD] Tweet', '[BOLD] 2013  [BOLD] SMS', '[BOLD] 2014  [BOLD] Tweet', '[BOLD] 2014  [BOLD] Tweet', '[BOLD] 2014  [BOLD] Live-', '[BOLD] 2015  [BOLD] Tweet', '[BOLD] 2016  [BOLD] Tweet']
We also tested the participating systems on the test sets from the three previous editions of this subtask. Participants were not allowed to use these test sets for training. The top-performing systems on Tweet 2016 are also top-ranked on the test datasets from previous years. There is a general pattern: the top-ranked system in year x outperforms the top-ranked system in year (x−1) on the official dataset of year (x−1). Top-ranked systems tend to use approaches that are universally strong, even when tested on out-of-domain test sets such as SMS, LiveJournal, or sarcastic tweets (yet, for sarcastic tweets, there are larger differences in rank compared to systems rankings on Tweet 2016). It is unclear where improvements come from: (a) the additional training data that we made available this year (in addition to Tweet-train-2013, which was used in 2013–2015), thus effectively doubling the amount of training data, or (b) because of advancement of learning methods.