Biomedical relation extraction with pre-trained language representations and minimal task-specific architecture
Table 3: Baseline results on the four pre-defined classes, as well as ‘No rel’ (the negative class). All terms used here as defined in Table 1.
['[EMPTY]', '[BOLD] P', '[BOLD] R', '[BOLD] F1', '[BOLD] Supp.']
We compare our model performance against a simple baseline that predicts the class label by sampling from the categorical distribution of labels as calculated from the training set. Given the strongly skewed class distribution (which has low entropy of 0.46 bits, compared to 2.32 bits for a 5-class uniform distribution, and is therefore highly predictable), this is a strong baseline, especially for metrics reported on frequent classes. As expected, all metrics can achieve high scores on the negative (and by far the largest) class, illustrating how misleading micro-averaging with large classes can be as an indicator of model performance. Some classes have zero scores, which is unsurprising given their very low support in the dev set.