Deep Representation Learning for Clustering of Health Tweets
TABLE II: Calinski-Harabasz scores for several conventional methods and proposed cae-based methods for 3 different clustering algorithms and 3 different number of clusters.
['Tweet Representation', 'Features', '10 clusters k-means', '10 clusters Ward', '10 clusters spectral', '20 clusters k-means', '20 clusters Ward', '20 clusters spectral', '50 clusters k-means', '50 clusters Ward', '50 clusters spectral']
Same table shows the number of features used for each method as well. Document-term matrix extracted by BoWs and tf-idf features result in a sparse matrix of 63,326×13,026 with a sparsity of 0.9994733. Similarly, concatenation of word embeddings result in a high number of features with 32×300=9,600 for word2vec, GloVe and fastText, 32×768=24,576 for BERT embeddings. In summary, the proposed method of learning representations of tweets with CAEs outperform all of the conventional algorithms. When representations are compared with Hotelling’s T2 test (multivariate version of t-test), every representation distribution learned by CAEs are shown to be statistically significantly different than every other conventional representation distribution with p<0.001. In addition, introducing the L2-norm constraint on the learned representations during training enhances the clustering performance further (again p<0.001 when comparing for example fastText+CAE vs. fastText+L2-CAE). This statement holds true for 3 different clustering algorithms (k-means, Ward, spectral) as well as for 3 different number of clusters. A Calinski-Harabasz score of 4,304 has been achieved with constrained representation learning by CAE for the experiment of 50 clusters formed by k-means clustering. The highest CH score achieved in the same experiment setting by conventional algorithms was 638 which was achieved by LDA applied of tf-idf features. Visualizations of t-SNE and UMAP mappings in This phenomena is not unexpected as k-means clustering is based on L2 distance as well. The difference in learning curves for regular and constrained CAE trainings is also expected. Constrained CAE training converges to local minimum slightly later than unconstrained CAE, i.e., training of L2-CAE is slightly slower than that of CAE due to the introduced contraint