On the Ground Validation of Online Diagnosis with Twitter and Medical Records
Table 4: Results from an analysis of variance of the area under the ROC curve for classifiers based on tweets from an individual’s social network. Factors are whether the data is from the user’s friends or followers, the number of keywords chosen and the classifier.
['[EMPTY]', 'Df', 'Sum Sq', 'F value', 'Pr(>F)']
We further analyse the strength of these classifiers by building each classifier using 10 fold cross validation and calculate their performance by measuring area under the ROC curve. We repeat this 100 times to generate a distribution of each classifier’s performance. We find that the choice in classifier and the length of the feature vector have a significant effect on performance. We find that classifiers that use tweets from accounts that follow the user are significantly better at diagnosing the user than classifiers that use tweets from accounts that the user follows. This may be because Twitter users often follow celebrities and news organizations – and celebrities and news organizations rarely follow personal Twitter accounts – which could introduce excess noise.