Results of the seventh edition of the BioASQ Challenge
Table 5: Average system ranks across the batches of the Task 7a. A hyphenation symbol (-) is used whenever the system participated in fewer than 4 tests in the batch. Systems with fewer than 4 participations in all batches are omitted.
['[BOLD] System', '[BOLD] Batch 1 MiF', '[BOLD] Batch 1 LCA-F', '[BOLD] Batch 2 MiF', '[BOLD] Batch 2 LCA-F', '[BOLD] Batch 3 MiF', '[BOLD] Batch 3 LCA-F']
According to [Demsar06] the appropriate way to compare multiple classification systems over multiple datasets is based on their average rank across all the datasets. On each dataset the system with the best performance gets rank 1.0, the second best rank 2.0 and so on. In case two or more systems tie, they all receive the average rank. Note, that the average ranks are calculated for the 4 best results of each system in the batch according to the rules of the challenge.