Cross-lingual Dependency Parsing with Unlabeled Auxiliary Languages
Table 2: Cross-lingual transfer performances (UAS%/LAS%, excluding punctuation) of the SelfAtt-Graph parser Ahmad et al. (2019) on the test sets. In column 1, languages are sorted by the word-ordering distance to English. (en-fr) and (en-ru) denotes the source-auxiliary language pairs. ‘†’ indicates that the adversarially trained model results are statistically significantly better (by permutation test, p < 0.05) than the model trained only on the source language (en). Results show that the utilization of unlabeled auxiliary language corpora improves cross-lingual transfer performance significantly.
['Lang', 'Multilingual Word Embeddings (en)', 'Multilingual Word Embeddings (en-fr)', 'Multilingual Word Embeddings (en-ru)', 'Multilingual BERT (en)', 'Multilingual BERT (en-fr)', 'Multilingual BERT (en-ru)']
The results demonstrate that the adversarial training with the auxiliary language identification task benefits cross-lingual transfer with a small performance drop on the source language. When multi-lingual embedding is employed, the performance significantly improves, in terms of UAS of 0.48 and 0.61 over the 29 languages when French and Russian are used as the auxiliary language, respectively. When richer multilingual representation technique like mBERT is employed, adversarial training can still improve cross-lingual transfer performances (0.21 and 0.54 UAS over the 29 languages by using French and Russian, respectively).