Identifying Compromised Accounts on Social Media Using Statistical Text Analysis
Table 5. Category assignment of manually evaluated accounts.
['Category', 'Count']
In our first step we categorize k=20 accounts with the highest probability of being compromised into six categories { news, advertisement/spam, re-tweet bot, compromised, regular users, unknown }. We find that most of these accounts belong to categories with high variation in language, i.e., news, advertisement/spam, re-tweet bot. One of the accounts is found to be compromised. The algorithm also misclassified seven user accounts as being compromised. One account was categorized as “unknown”, since it was posting in a language that none of the evaluators speaks. It is reasonable to assume that if our algorithm were applied at much larger scale to all the Twitter users, it would most likely be able to detect many more compromised accounts. Furthermore, these results show that our algorithm can also be used to detect “unusual” accounts and users, thus potentially enabling development of novel text mining algorithms for analyzing user behavior on social media, which should be a very interesting future research topic. We meet this problem because our method uses a similarity function, in this case the KL-divergence, as the measurement to determine whether an account is being compromised. One common characteristic of news accounts and advertisement accounts is that they rarely use the same words or sentence patterns because they discuss different news and products. Thus, the high language diversity in these accounts results in the KL-divergence between any two time periods to be high, which leads to the misclassification.