Overview of CheckThat! 2020: Automatic Identification and Verification of Claims in Social Media
Table 2: Performance of the best run per team for Arabic Task 1.
['[BOLD] RunID', '[ITALIC] P@10', '[ITALIC] P@20', '[ITALIC] P@30', '[BOLD] MAP']
S3SS1SSS0Px3 Evaluation We treated Task 1 as a ranking problem where we expected check-worthy tweets to be ranked at the top. We evaluated the runs using precision at k (P@k) and Mean Average Precision (MAP). We considered P@30 as the official measure, as we anticipated the user would check maximum of 30 claims per week. We also developed two simple baselines: baseline 1, which ranks tweets in descending order based on their popularity score (sum of likes and retweets a tweet has received) and baseline 2, which ranks tweets in reverse chronological order, i.e., most-recent first. We can see that most teams managed to improve over the two baselines by a large margin.