On the Ground Validation of Online Diagnosis with Twitter and Medical Records
Table 3: Confusion matrix of the classifier based on anomalous tweeting rates. Rows are of true values, columns are of predicted values.
['Sick', 'Not Sick', '[EMPTY]']
Where ¯x We repeat this process for months when the user is not sick. We then classify the user as sick if z>1.411 where 1.411 was chosen through leave one out cross validation. We find a significant difference between the z-scores for months when a user had influenza and months when the user did not (p=0.01303, two-sample Kolmogorov-Smirnov test). Most of the time individuals are not sick (219 / 258 = 84.88% of the months), resulting in a highly biased sample. Thus we optimize based on the F1 score instead of accuracy. The optimal z-score cutoff results in an area under the ROC curve of .6218 and F1=35.0%.