How multilingual is Multilingual BERT?
(a) Subj./verb/obj. order.
['[EMPTY]', 'SVO', 'SOV']
The results reported include only zero-shot transfer, i.e. they do not include cases training and testing on the same language. We can see that performance is best when transferring between languages that share word order features, suggesting that while M-Bert’s multilingual representation is able to map learned structures onto new vocabularies, it does not seem to learn systematic transformations of those structures to accommodate a target language with different word order.