Lexical Bias In Essay Level Prediction
Table 4: Ablation study to explore the importance of different feature families.
['Features', 'Error ( [ITALIC] E)', 'Accuracy']
The Table presents the scores achieved when using all features as well as the scores achieved when removing a particular family of features. In this sense, one can estimate the added value of each family. From the table we notice that the most effective features are the numerical features, the document distributions learned with LDA and the scores from the language models. When removing the features of this family we notice the biggest reduction in performance. For reference, the table presents the performance of a tuned Logistic Regression with the same class weights for the class imbalance. Gradient Boosted trees outperform Logistic Regression by a large margin in terms of E, proving their efficiency for classification tasks. Notice, for instance, that while language models are quite important for the custom error metric E (Eq. This suggests that adding them reduces the size of the errors considerably, but does not increase much the correctly classified instances.