Measuring Domain Portability and Error Propagation in Biomedical QA
Table 2: Performance on BioASQ Task 7b, Phase B for batches 3 to 5. The underlined system is the best scoring system from Google’s entries and bold indicates the system is the top from all official entries in that batch. Best Score is the top scoring entry that is not among Google’s submissions.
['[EMPTY]', 'Batch 3 Strict', 'Batch 3 Lenient', 'Batch 3 MRR', 'Batch 4 Strict', 'Batch 4 Lenient', 'Batch 4 MRR', 'Batch 5 Strict', 'Batch 5 Lenient', 'Batch 5 MRR']
The table alludes to a number of interesting results some we discuss in later subsections. One of those results is that adding abstracts was not significantly helpful and indicates that there is a noise-to-signal issue where the system might get diminishing or negative gains after a certain amount of data is used for the context.