Cross-lingual Dependency Parsing with Unlabeled Auxiliary Languages
Table 5: Comparison between adversarial training (AT) and multi-task learning (MTL) of the contextual encoders. Columns 2–5 demonstrate the parsing performances (UAS%/LAS%, excluding punctuation) on the auxiliary languages and average of the 29 languages. Columns 6–7 present accuracy (%) of the language label prediction test. ‘†’ indicates that the performance is higher than the baseline performance (shown in the 2nd column of Table 2).
['Lang (Src. + Aux.)', 'Auxiliary Language Perf. AT', 'Auxiliary Language Perf. MTL', 'Average Cross-lingual Perf. AT', 'Average Cross-lingual Perf. MTL', 'Lang. Test Perf. AT', 'Lang. Test Perf. MTL']
Interestingly, although the MTL objective sounds contradiction to adversarial learning, it has a positive effect on the cross-lingual parsing, as the representations are learned with certain additional information from new (unlabeled) data. The comparisons on parsing performances do not reveal whether the contextual encoders learn to encode language-agnostic or dependent features. Therefore, we perform language test with the MTL and AT The results indicate that the MTL encoders consistently perform better than the AT encoders, which verifies our hypothesis that adversarial training motivates the contextual encoders to encode language-agnostic features.