Automatic Classification of the Complexity of Nonfiction Texts in Portuguese for Early School Years
Table 3: Selection of texts via Active Learning and accuracy obtained from SVM
['[BOLD] Step', '[BOLD] Texts', '[BOLD] Accuracy']
We performed four steps to select texts for annotation, where each step selected the 100 most complex texts for SVM. The texts that could not be processed due to parsing problems were removed. They show that even when we select the texts in which the classifier has greater indecision in classifying, the SVM has not yet been able to define a boundary between the classes, which led to lower accuracy in classifying data. This shows that there is a mix between classes so that the 108 current features are not able to correctly distinguish the five levels manually annotated. Finally, we conducted a stage of selecting the 100 most easily annotated texts (those with greater distance from SVM separating hyperplanes) in order to contrast with the current distribution of data and the accuracy obtained. We obtained a set of 1,456 texts with the following distribution: 242 texts of level 1, 313 texts of level 2, 338 texts of level 3,287 texts of level 4 and 276 texts of level 5. The accuracy obtained when performing a 10-fold cross-validation experiment with linear kernel SVM and C=1 was 52% (+/- 15).