eRevise: Using Natural Language Processing to Provide Formative Feedback on Text Evidence Usage in Student Writing
Table 3: Quadratic Weighted Kappa (QWK) of different AES models. The CO-ATTN model significantly outperforms the Rubric and SG models, respectively (p<0.05).
['[BOLD] AES Model', '[BOLD] QWK']
We have developed several AES systems for RTA assessment Our first model (denoted by Rubric) A subsequent model (denoted by SG) Most recently, \citeauthorzhang2018co \shortcitezhang2018co developed a neural network model with a co-attention layer (denoted by CO-ATTN) to eliminate human feature engineering. Although the neural CO-ATTN model has the best performance, to select formative feedback messages that address essay weaknesses in terms of rubric constructs, a more interpretable representation of the essay is necessary. Therefore, SG is the AES system used in eRevise. In particular, two of the features used by SG for score prediction, namely Number of Pieces of Evidence (NPE) and Specificity (SPC), form the basis of eRevise’s feedback selection algorithm.