Why Attention? Analyze BiLSTM Deficiency and Its Remedies in the Case of NER
Table 1: Overall results. *Used on WNUT for character-based word vectors, reported better than CNN.
['[EMPTY]', 'OntoNotes 5.0 Prec.', 'OntoNotes 5.0 Rec.', 'OntoNotes 5.0 F1', 'WNUT 2017 Prec.', 'WNUT 2017 Rec.', 'WNUT 2017 F1']
The models proposed in this paper surpassed previous reported bare-bone models by 1.4% on OntoNotes and 4.6% on WNUT. Compared to the re-implemented Baseline-BiLSTM-CNN, the cross-structures brought 0.7% and 2.2% improvements on OntoNotes and WNUT. More substantial improvements were achieved for WNUT 2017 emerging NER, suggesting that cross-context patterns were even more crucial for emerging contexts and entities than familiar entities, which might often be memorized by their surface forms.