Effective sampling for large-scale automated writing evaluation systems
Table 1: ASAP AES rubric
['Set', 'Task', 'Grade', 'Range']
An AWE data set consists of some number of written responses and a set of scores for each response. In our experiments we use the industry track data from the Automated Student Assessment Prize (ASAP) Automated Essay Scoring (AES) competition sponsored by the Hewlett Foundation held in 2012 It contains training and test sets numbered 1 through 8. The target variable of the data sets is the score or sum of scores that an essay was given by one or more human raters. Essay set 2 has two target variables, which we identify as 2a and 2b depending on which target variable the model was trained to predict. For all essay sets the target variable includes all integers between the minimum and maximum scores. The target variable for essay set 7, for example, is the integers 2, 3, …, 30.