Sequence Tagging with Contextual and Non-Contextual Subword Representations: A Multilingual Evaluation
Table 5: NER F1 scores for the 101 WikiAnn languages supported by all evaluated methods.
['Languages', 'Pan17', 'FastText', 'BPEmb +char', 'MultiBPEmb +char+finetune', 'BERT', 'BERT +char', 'BERT +char+BPEmb']
Multilingual BERT outperforms the Pan17 baseline and shows strong results in comparison to monolingual BPEmb. The combination of multilingual BERT, monolingual BPEmb, and character embeddings is best overall (92.0) among models trained only on monolingual NER data. However, this ensemble of contextual and non-contextual subword embeddings is inferior to MultiBPEmb (93.2), which was first trained on multilingual data from all languages collectively, and then separately finetuned to each language.