UnibucKernel: A kernel-based learning method for complex word identification
Table 2: Classification results on the three data sets of the English monolingual track of the 2018 CWI Shared Task. The methods are evaluated in terms of the classification accuracy and the F1-score. The results marked with an asterisk are obtained during the competition. The other results are obtained after the competition.
['Data Set English News', 'Kernel linear', 'Accuracy 0.8653', '[ITALIC] F1-score 0.8547', '[ITALIC] F1-score 0.8111∗', 'Competition Rank 12', 'Post-Competition Rank 6']
Results. We would like to note that, before the competition ended, we observed a bug in the code that was used in most of our submissions. In the feature extraction stage, the code produced NaN (not a number) values for some features. In order to make the submissions in time, we had to eliminate the samples containing NaN values in the feature vector. Consequently, most of our results during the competition were lower than expected. However, we managed to fix this bug and recompute the features in time to resubmit new results, but only for the RBF kernel on the English Wikipedia data set. Nevertheless, for a fair comparison with the other systems, we include our F1-scores and rankings during the competition as well as the post-competition F1-scores and rankings. Our best F1-score on the English News data set is 0.8594, which is nearly 1.4% lower than the top scoring system, which attained 0.8736 during the competition. On the English WikiNews data set, our best F1-score (0.8201) is once again about 2% lower than the top scoring system, which obtained 0.8400 during the competition. On the English Wikipedia data set, our best F1-score is 0.7919. With this score, we ranked as the third team on the English Wikipedia data set. Two systems performed better on English Wikipedia, one that reached the top F1-score of 0.8115 and one that reached the second-best scored of 0.7965. Overall, our system performed quite well, but it can surely benefit from the addition of more features.