Mining Supervisor Evaluation and Peer Feedback in Performance Appraisals
Table 1: Results of 5-fold cross validation for sentence classification on dataset D1.
['[BOLD] Classifier', 'STRENGTH  [BOLD] P', 'STRENGTH  [BOLD] R', 'STRENGTH  [BOLD] F', 'WEAKNESS  [BOLD] P', 'WEAKNESS  [BOLD] R', 'WEAKNESS  [BOLD] F', 'SUGGESTION  [BOLD] P', 'SUGGESTION  [BOLD] R', 'SUGGESTION  [BOLD] F', '[BOLD] A']
We randomly selected 2000 sentences from the supervisor assessment corpus and manually tagged them (dataset D1). This labelled dataset contained 705, 103, 822 and 370 sentences having the class labels STRENGTH, WEAKNESS, SUGGESTION or OTHER respectively. We trained several multi-class classifiers on this dataset. For the first 5 classifiers, we used their implementation from the SciKit Learn library in Python (scikit-learn.org). The features used for these classifiers were simply the sentence words along with their frequencies. The overall accuracy for a classifier is defined as A=#correct_predictions#data_points, where the denominator is 2000 for dataset D1. Note that the pattern-based approach is unsupervised i.e., it did not use any training data. Hence, the results shown for it are for the entire dataset and not based on cross-validation.