Predicting Annotation Difficulty to Improve Task Routing and Model Performance for Biomedical Information Extraction
Table 5: Interventions IE model performance trained crowd or expert. The first four models are trained with a subset of 1k abstracts and the base model is trained with all 5k abstracts.
['[EMPTY]', '[BOLD] Precision', '[BOLD] Recall', '[BOLD] F1']
The first two rows show the results for models trained with expert annotations. The model trained on random data has a slightly better F1 than that trained on the same amount of difficult data. The model trained on random data has higher precision but lower recall.