A German Corpus for Fine-Grained Named Entity Recognition and Relation Extraction of Traffic and Industry Events
Table 6: Performance of a standard CRF-based NER classifier on the presented dataset.
['[BOLD] Evaluation Type', '[BOLD] Precision', '[BOLD] Recall', '[BOLD] F1']
For NER, we evaluated the model’s performance both at the token level and at the concept level. We see that the overall token-level F1 score is close to 0.85, a respectable figure given the confusability of location subtypes such as streets, stops, and routes in our dataset. For organization and organization-company entities, the average token-level F1 score is lower at approximately 0.78, but for location and its subtypes, it lies between 0.85−0.92 (not shown).