Assessing State-of-the-Art Sentiment Models on State-of-the-Art Sentiment Datasets
Table 3: Accuracy on the test sets. For all neural models we perform 5 runs and show the mean and standard deviation. The best results for each dataset is given in bold and results that have been previously reported are highlighted. All results derive from our reimplementation of the methods. We describe significance values in the text and appendix. Footnotes refer to the work where a method was previously tested on a specific dataset, although not necessarily with the same results: [1] Tai et al. (2015) [2] Kim (2014) [3] Faruqui et al. (2015) [4] Lambert (2015) [5] Uryupina et al. (2014) [6] Tang et al. (2014).
['[EMPTY]', 'Model', 'Dim.', 'SST-fine', 'SST-binary', 'OpeNER', 'SenTube-A', 'SenTube-T', 'SemEval', 'Macro-Avg.']
Since the reported accuracies for the neural models are the means over five runs, we cannot use this technique in a straightforward manner. Datasets where there is a statistical difference (above diagonal) and number of datasets where a model on the Y axis is statistically better than a model on the X axis (below diagonal).