The Secret Lives of Names? Name Embeddings from Social Media
Table 4. Nationality classification performances (f1 scores) of Email (Em.) and Twitter (aggregated*, Tw.) embeddings on a 39-leaf nationality taxonomy. The taxonomy has three levels, which are separated with bolder lines. ‘*’ marks leaf nationalities. Weighted Avg. is count-weighted average F1 score of leaf nationalities. Twitter embeddings achieve comparable performances on nationality classification.
['Nationality CelticEnglish*', 'Name# 3505K', '[ITALIC] Em. 0.73', '[ITALIC] Tw. 0.70', 'Nationality Muslim', 'Name# 1475K', '[ITALIC] Em. 0.74', '[ITALIC] Tw. 0.73', 'Nationality Jewish*', 'Name# 11K', '[ITALIC] Em. 0.40', '[ITALIC] Twi. 0.37']
(Tw.) has similar performance as Email (Em.). For some classes, like Thailand, Baltic and Pakistan, Aggregated* outperforms Email embeddings. Email performs slightly better than Twitter w.r.t. weighted average F1 score on 39 leaf classes. We also noticed that the performances are highly dependent on the size of data. For less developed places like Cambodia and countries in central Asian and Maghreb, very limited user associations and labels are collected. Therefore, their F1 scores are much below average performance.