On the Ground Validation of Online Diagnosis with Twitter and Medical Records
Table 2: Confusion matrix of a Tweet-Classification based diagnosis system. Rows are of true values, columns are of predicted values.
['Sick', 'Not Sick', '[EMPTY]']
Second, we consider analysing the content of a tweet’s text for messages giving hints about being sick such as “another doctor’s appointment Wednesday … have to #treatmyflu” or Computational approaches for natural language processing are available. However, because our dataset is relatively small, we use a ‘human’ classifier by hand rating all 1609 tweets that were posted by individuals during the time of their illness. We also sample a randomly selected set of 1609 tweets from times when the users did not have influenza as a control. We find 58 tweets from 17 (17/35=48.57%) individuals in our study that are about the user being sick. We also find zero tweets about the user having influenza during times when they did not have influenza. Because humans are very good at extracting information from text, hand rating tweets allows for an approximately 100.0% accurate classification, although it clearly does not scale well. Extracting information from text using machine learning is a complex problem where finding solutions that perform as well as humans is rare.