Overview of CheckThat! 2020: Automatic Identification and Verification of Claims in Social Media
Table 8: Task 2, English: performance for the primary submissions and for an Elastic Search (ES) baseline.
['[BOLD] Team', '[BOLD] MAP  [BOLD] @1', '[BOLD] MAP  [BOLD] @3', '[BOLD] MAP  [BOLD] @5', '[BOLD] MAP  [BOLD] –', '[BOLD] Precision  [BOLD] @1', '[BOLD] Precision  [BOLD] @3', '[BOLD] Precision  [BOLD] @5', '[BOLD] RR  [BOLD] @1', '[BOLD] RR  [BOLD] @3', '[BOLD] RR  [BOLD] @5']
S4SS2SSS0Px3 Evaluation The official evaluation measure for Task 2 is MAP@k for k=5. However, we further report MAP for k∈{1,3,10,20}, overall MAP, R-Precision, Average Precision, Reciprocal Rank, and Precision@k. We can see that the winner Buster. AI and the second-best UNIPI-NLE are well ahead of the remaining teams by several points absolute on all evaluation measures. We can further see that most systems managed to outperform an Elastic Search (ES) baseline by a huge margin. The data and the evaluation scripts are available online.