Cross-Lingual BERT Transformation for Zero-Shot Dependency Parsing
Table 1: Results (LAS%) on test sets. Languages are split by language families with dashed lines. AVG. means the average of results from all target languages. Statistically significant differences between our methods and the mBERT model are marked with *, with p-value 
['[BOLD] Lan.', 'Static  [BOLD] FT-SVD', 'Contextualized  [BOLD] mBERT', 'Contextualized  [BOLD] CLBT (SVD)', 'Contextualized  [BOLD] CLBT (GD)']
Results on the test sets are shown in Table Overall, our approach with either SVD or GD outperforms both FT-SVD and mBERT by a substantial margin (+2.91% in averaged LAS), among which GD turns out to be slightly better than SVD in most of the languages. When combined with FT-SVD, the performances can be further improved by 0.33% in LAS for the GD method and 0.51% for SVD (see the Appendix for more details). Interestingly, the mBERT model which is trained without any cross-lingual supervision but using a shared multilingual wordpiece vocabulary works surprisingly well in some languages, especially in those linguistically close to English. Similar observations have also been identified in other works pires2019multilingual; wu2019beto.