RSDD-Time: Temporal Annotation of Self-Reported Mental Health Diagnoses
Table 3: Macro-averaged classification results for diagnosis recency and condition state using tf-idf vectorized features for various baseline models.
['[EMPTY]', '[BOLD] Diagnosis Recency  [BOLD] P', '[BOLD] Diagnosis Recency  [BOLD] R', '[BOLD] Diagnosis Recency  [BOLD] F1', '[BOLD] Condition State  [BOLD] P', '[BOLD] Condition State  [BOLD] R', '[BOLD] Condition State  [BOLD] F1']
We train several models to classify diagnosis recency and condition state. In each we use basic bag-of-character-ngrams features. Character ngrams of length 2-5 (inclusive) are considered, and weighted using tf-idf. To account for class imbalance, samples are weighed by the inverse frequency of their category in the training set. We compare three models: logistic regression, a linear-kernel Support Vector Machine (SVM), and Gradient-Boosted ensemble Trees (GBT) Chen and Guestrin The logistic regression and SVM models are ℓ2 normalized, and the GBT models are trained with a maximum tree depth of 3 to avoid overfitting. The GBT method performs best for diagnosis recency classification, and logistic regression performs best for condition state classification. This difference could be due to differences in performance because of skew. The condition state data is more skewed, with current and don’t know accounting for almost 80% of the labels.