TENER: Adapting Transformer Encoder for Named Entity Recognition
Table 2: The F1 scores on Chinese NER datasets. ♣,♠ are results reported in (Zhang and Yang, 2018) and (Gui et al., 2019a), respectively. “w/ scale” means TENER using the scaled attention in Eq.(20). ∗ their results are not directly comparable with ours, since they used 100d pre-trained character and bigram embeddings. Other models use the same embeddings.
['Models', 'Weibo', 'Resume', 'OntoNotes4.0', 'MSRA']
However, when relative positional encoding combined, the performance was enhanced greatly, resulting in better results than the BiLSTM and CNN in all datasets. The number of training examples of the Weibo dataset is tiny, therefore the performance of the Transformer is abysmal, which is as expected since the Transformer is data-hungry. Nevertheless, when enhanced with the relative positional encoding and unscaled attention, it can achieve even better performance than the BiLSTM-based model. The superior performance of the adapted Transformer in four datasets ranging from small datasets to big datasets depicts that the adapted Transformer is more robust to the number of training examples than the vanilla Transformer.