Sequence Tagging with Contextual and Non-Contextual Subword Representations: A Multilingual Evaluation
Table 3: NER results on WikiAnn. The first row shows macro-averaged F1 scores (%) for all 265 languages in the Intersect. 1 setting. Rows two to four break down scores for 188 low-resource languages (<10k instances), 48 medium-resource languages (10k to 100k instances), and 29 high-resource languages (>100k instances).
['Languages', 'Pan17', 'FastText', 'BPEmb', 'BPEmb +char', 'BPEmb +shape', 'BPEmb +someshape', 'MultiBPEmb+char -finetune', 'MultiBPEmb+char +finetune']
In this section, we evaluate FastText and BPEmb on NER in 265 languages. As baseline, we compare to Pan et al. Averaged over all languages, FastText performs 4.1 F1 points worse than this baseline. BPEmb is on par overall, with higher scores for medium- and high-resource languages, but a worse F1 score on low-resource languages. BPEmb combined with character embeddings (+char) yields the overall highest scores for medium- and high-resource languages among monolingual methods. Word shape. that on the one hand reduces vocabulary size, but on the other loses information in writing systems with a distinction between upper and lower case letters. For example, Magnus is mapped to the word shape Aa and G.M. to A.A. These results show that multilingual training followed by monolingual finetuning is an effective method for low-resource sequence tagging.