Using Noisy Self-Reports to Predict Twitter User Demographics
Table 2: Applying our four HF filters (§ 4) individually and altogether. Precision is calculated on our manually-annotated dev set from Appendix B, after thresholding on self-report score.
['[EMPTY]', 'Raw', 'Color', 'Plural', 'Bigram', 'Quote', 'All']
As the QB dataset contains only 112k users, we now consider a less restrictive approach. Our second dataset uses four heuristic filters applied to the original 2.67M users that discard common false positives. Second, as racial keywords followed by plural nouns were unlikely to be self-reports (e.g. “white people”) we used NLTK TweetTokenizer Bird et al. Third, we curate a list of bigrams that frequently contain a query but are unlikely to be self-reports (e.g. “black sheep,”). This produces an intersection of 286 non-self-report bigrams, which we filter out. Finally, we ignore query words that appear inside quotation marks. Applying all four filters gives a total of 1.72M users, and after thresholding on the self-report score, this dataset is reduced to 228k users. Of these, only 135k users could be scraped in 2019. We call this dataset Heuristic-Filtered (HF).