Improving the results of string kernels in sentiment analysis and Arabic dialect identification by adapting them to your test set
Table 3: Arabic dialect identification accuracy rates (in %) of our adapted string kernels versus the 2017 ADI Shared Task winner (sota) Ionescu and Butnaru (2017) and the first runner up Malmasi and Zampieri (2017). The best accuracy rates are highlighted in bold. The marker * indicates that the performance is significantly better than Ionescu and Butnaru (2017) according to a paired McNemar’s test performed at a significance level of 0.01.
['Method', 'Development', 'Test']
Results. The domain-adapted sum of kernels obtains improvements above 0.8% over the state-of-the-art sum of kernels Ionescu and Butnaru The improvement on the development set (from 64.17% to 65.42%) is statistically significant. Nevertheless, we obtain higher and significant improvements when we employ the transductive classifier. Our best accuracy is 66.73% (2.56% above the baseline) on the development set and 78.35% (2.08% above the baseline) on the test set. The results show that our domain adaptation framework based on string kernels attains the best performance on the ADI Shared Task data set, and the improvements over the state-of-the-art are statistically significant, according to the McNemar’s test.