Depression and Self-Harm Risk Assessment in Online Forums
Table 7: Self-harm risk assessment performance on the ReachOut CLPsych ’17 test set. All methods perform substantially worse than on the CLPsych ’16 test data. The difference in performance between the following method pairs are statistically significant (McNemar’s test, p<0.05): Categorical Cross Ent. and MSE, and MSE and Class Metric (Ordinal).
['Method', 'Non-green F1', 'Flagged F1', 'Flagged Acc.', 'Urgent F1', 'Urgent Acc.', 'All F1', 'All Acc.']
All methods perform substantially worse than they performed on the CLPsych ’16 test data as measured by non-green, urgent, and overall F1. The trends across methods remain similar, however, with Categorical Cross Ent. performing the best as measured by non-green and overall F1, and with no statistically significant difference between Class Metric (Ordinal) and the best performing method.