AUEB at BioASQ 6: Document and Snippet Retrieval
Table 1: Performance on BioASQ Task 6b, Phase A (batches 1–5) for document and snippet retrieval (left and right tables, respectively). Systems described in Section 5.3. The italicised system is the top scoring system from AUEB’s entries and if also in bold, is the top from all official entries in that batch. Top is by map, the official metric of BioASQ. Top Competitor is the top scoring entry – by map– that is not among AUEB’s submissions.
['[BOLD] System', '[BOLD] F1', '[BOLD] MAP', '[BOLD] GMAP']
There are a number of things to note. First, for document retrieval, there is very little difference between our submitted models. Both pacrr- and drmm-based models perform well (usually at the top or near the top) with less than 1 map point separating them. These systems were all competitive and for 4 of the 5 batches one was the top scoring system in the competition. On average the experimental abel-drmm system (aueb-nlp-4) scored best amongst AUEB submissions and in aggregate over all submissions, but by a small margin (0.1053 average map versus 0.1016 for term-pacrr). The exception was the high precision system (aueb-nlp-5) which did worse in all metrics except F1, where it was easily the best system for the 4 batches it participated in. This is not particularly surprising, but impacted snippet selection, as we will see.