Item Recommendation with Evolving User Preferences and Experience1
TABLE VIII: Performance on identifying experienced users.
['[BOLD] Models', '[ITALIC] F1', '[ITALIC] NDCG']
Identifying Experienced Users: Our second task is to find experienced members of this community, who have potential for being citizen journalists. In order to find how good our model predicts the experience level of users, we consider the following as ground-truth for user experience. In NewsTrust, users have Member Levels calculated by the NewsTrust staff based on community engagement, time in the community, other users’ feedback on reviews, profile transparency, and manual validation. We use these member levels to categorize users as experienced or inexperienced. We also computed the Normalized Discounted Cumulative Gain (NDCG) NDCG gives geometrically decreasing weights to predictions at various positions of ranked list: NDCGp=DCGpIDCGp where DCGp=rel1+∑pi=2relilog2i