On the Choice of Auxiliary Languages for Improved Sequence Tagging
Table 2: Comparison with state of the art.
['NER', 'ner/Strakova19', 'De  [BOLD] 85.1', 'En  [BOLD] 93.3', 'Es  [BOLD] 88.8', 'Fi -', 'Nl  [BOLD] 92.7']
For meta-embeddings, we found statistically significant differences in 12 out of 20 settings (6 with BPEmb, 6 with FLAIR) against the best monolingual + X model, while we found statistically significant differences for Mono + All in only 7 out of 20 cases. This suggests that meta-embeddings are superior to monolingual settings with one auxiliary language as well as to the concatenation of all embeddings. Further, we found that the combination of monolingual and multilingual byte-pair encoding embeddings is always superior to either monolingual or multilingual embeddings alone for both tasks. Even though the multilingual embeddings have seen many languages during pre-training, they can still benefit from the high performance of monolingual embeddings and vice versa. As the meta-embeddings assign attention weights for each embedding, we can inspect the importance the models give to the different embeddings.