Unicoder: A Universal Language Encoder by Pre-training with Multiple Cross-lingual Tasks
Table 4: Experiments of fine-tuning on different number of languages. The model is evaluated on 6 languages, and the average result is at last column. The results in the last row correspond to the results in last row of Table 2.
['language', 'XNLI-en', 'XNLI-ar', 'XNLI-es', 'XNLI-fr', 'XNLI-ru', 'XNLI-zh', 'average']
In most languages, we could find that the more languages we used in fine-tuning, the better the performance. Chinese and Russian have two numbers don’t follow this trend. But 15 languages always outperform 1 language for each languages.