Rethinking Generalization of Neural Models: A Named Entity Recognition Case Study
Table 5: The test performance (F1 score) on CoNLL 2003 and its revised version.
['[BOLD] Model', '[BOLD] CoNLL', '[BOLD] ReCoNLL']
The results are shown in Tab. We find that once these errors are fixed, the performance of all these models has been improved, which indicates that human annotation errors cover the actual generalization ability of the existing model. Notably, the NER model FLAIR has driven the state-of-the-art result to a new level.