Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERT
Table 6: Dependency parsing results by language (UAS/LAS). * denotes delexicalized parsing in the baseline. S and Z denotes supervised learning and zero-shot transfer. Bold and underline denotes best and second best. We order the languages by word order distance to English.
['[EMPTY]', 'Dist', 'mBERT(S)', 'Baseline(Z)', 'mBERT(Z)', 'mBERT(Z+POS)']
Note in practice, gold POS tags are not always available, especially for low resource languages. Interestingly, the LAS of mBERT tends to weaker than the baseline in languages with less word order distance, in other words, more closely related to English. With the help of gold POS tags, we further observe 1.6 points UAS and 4.7 point LAS absolute improvement on average. It appears that adding gold POS tags, which provide clearer cross-lingual representations, benefit mBERT.