Text-based depression detection on sparse data
TABLE VII: Comparison between Time, Mean, Attention and Max pooling methods regarding four utilizied features. The global average values μ(ρ) for each metric ρ are displayed.
['Data', 'Pooling', 'Embedding', 'MAE', 'F1', 'Acc']
In this experiment we aim to ascertain the usage of the four proposed pooling functions, namely Attention, Max, Mean, and Time pooling. We report our findings (metric ρ) as a global average (μ) across all runs, independent of folds: μ(ρ)=1FKF=5∑f=1K=100∑k=1rkf(ρ) Time-pooling can be seen to be the least favorable pooling method, regardless of which text embedding is utilized. The other three pooling methods’ performance are slightly different depending on the front-end embeddings. Across all utilized pooling functions, ELMo is seen to provide consistently well performing results regarding F1 and MAE. Utilizing ELMo with attention, mean or max pooling results in an average MAE of ≈5.06. Since ELMo is pretrained on a larger dataset (1 Billion word) compared to the other embeddings, we believe that this is the main reason for the performance gains on the development set. BERT embeddings, while outperforming ELMo ones on the cv dataset , degrade on the development set. Interestingly, Word2Vec embeddings with mean-pooling are observed to produce the lowest MAE error in the held-out cv set. We conclude from the set of experiments, that the pooling functions are essential, since by choosing the naive time-pooling strategy, performance significantly drops. Mean, attention and max pooling all seem to be a viable approach for depression detection. As can be seen, the classification performance difference between all four utilized features is marginal. This is likely due to our multi-task training scheme and improved data balancing pipeline, such that the model is prevented from overfitting towards depressed/healty patients.