Cross-Lingual Ability of Multilingual BERT: An Empirical Study
Table 2: Contribution of Word-Ordering similarity: We study the importance of word-order similarity by analysing the performance of XNLI and NER when some percent of word-order similarity is reduced. The percent p controls the amount of similarity (how random each sentence is permuted). We can see that word-order similarity is quite important, however there must be other components of structural similarity that could contribute for the cross-lingual ability, as the performance of almost random is still passable.
['[BOLD] B-BERT', '[BOLD] Permutation amount', '[BOLD] XNLI (acc)', '[BOLD] NER (F{}_{1}-Score)']
However, the cross-lingual performance is still significantly better than random, which indicates that there are other components of structural similarity, which could contribute to the cross-lingual ability of B-BERT.