Evaluating the Cross-Lingual Effectiveness of Massively Multilingual Neural Machine Translation
Table 8: F1 scores on dev set of POS tagging dataset in few-shot setting using 10 examples from each language in addition to English data.
['ar', 'mBERT 85.20', 'MMTE  [BOLD] 85.53', 'nl', 'mBERT 90.01', 'MMTE  [BOLD] 90.26']
S4SS0SSS0Px2 Few Shot Transfer While zero-shot transfer is a good measure of a model’s natural cross-lingual effectiveness, the more practical setting is the few-shot transfer scenario as we almost always have access to, or can cheaply acquire, a small amount of data in the target language. To simulate the few-shot setting, in addition to using English data, we use 10 examples from each language (upsampled to 1000). MMTE outperforms mBERT in few-shot setting by 0.6 points averaged over 48 languages. Once again, we see that the gains are more pronounced in low resource languages.