Heuristic Feature Selection for Clickbait Detection
Table 2: Features by category with a leave-many-out score >10−5. All scores are multiplied by 10−4. Feature categories shown are character n-grams (CNG), word n-grams (WNG), word list features, and engineered features.
['[BOLD] Engineered', '[BOLD] Score']
All feature types include good and bad features with differences in strength and frequency. Most of the 12 engineered features have a high impact about one order of magnitude larger than that of any other feature. However, two of the engineered features harm prediction performance (the average word length and whether a tweet starts with a number). One reason for the relatively high impact of the engineered features is that they are not as sparse as the n-grams or word lists. Interestingly, word n-grams are the only feature category without high-impact confusing features. An interesting side note is that the question mark seems to be discriminatory as a word token, but confusing as a character token, while the exclamation mark is discriminatory in both categories. Maybe more compact models could thus be constructed if the information of individual features with high impact could be combined into engineered features with higher impact. An example for a new engineered feature could be something like the “number of words ending with -ly or -ing” (counting adverbs, gerunds, etc.) since these suffixes seem to be rather discriminating character n-gram features. However, the hypothesis of creating good engineered features from individual other features is only partially supported by a qualitative analysis of the high-impact features. While the engineered “number of stopwords” feature is very discriminatory and most of the high-impact word 1-grams are actually stop words, the engineered “number of mentions” feature (counting the number of @ ’s in a tweet) is not in the table of the high-impact engineered features, even though 3 of the 18 most discriminatory character tokens involve the ’@’ character.