Rethinking Generalization of Neural Models: A Named Entity Recognition Case Study
Table 6: Illustration of F1 score, EECR, and CCR on cross-dataset setting. P-row and P-col represent row- and column-wise Pearson correlation coefficient. Green, Pink and Yellow regions denote the correlation between MF1 and Mρ+Mϕ, Mρ, Mϕ respectively. The Blue is the overall correlation coefficient.
['[BOLD] Matrix', '[BOLD] Train', '[BOLD] WN.', '[BOLD] Co.', '[BOLD] BN', '[BOLD] WB', '[BOLD] MZ', '[BOLD] NW', '[BOLD] BC', '[BOLD] P-row']
Each column corresponds to the performance when testing on one dataset and training on each of other datasets. We detail our findings as follows: A significant case can be found in Tab. We can speculate the reason from the ϕ-M, that the contextual coverage ratio between WB and MZ is much higher than utilizing WNUT and CoNLL as training sets. The above results show that the generalization ability of NER models is influenced not only by whether the test entity has been seen in the training set but also by whether the context of the test entity has been seen.