Cross-lingual Dependency Parsing with Unlabeled Auxiliary Languages
Table 4: Average cross-lingual transfer performances (UAS%, excluding punctuation) on the test sets using different adversarial training objective and setting. Multilingual word embeddings are used for these experiments.
['AT', 'SelfAtt-Graph en-fr', 'SelfAtt-Graph en-fr', 'SelfAtt-Graph en-ru', 'SelfAtt-Graph en-ru', 'RNN-Stack en-fr', 'RNN-Stack en-fr', 'RNN-Stack en-ru', 'RNN-Stack en-ru']
To understand the impact of different adversarial training types and objectives, we apply adversarial training on both word- and sentence-level with gradient reversal (GR), GAN, and WGAN objectives. Among the adversarial training objectives, we observe that in most cases, the GAN objective results in better performances than the GR and WGAN objectives. Our finding is in contrast to Adel et al. To further investigate, we perform the language test on the encoders trained via these two objectives. We find that the GR-based trained encoders perform consistently better than the GAN based ones on the language identification task, showing that via GAN-based training, the encoders become more language-agnostic. In a comparison between GAN and WGAN, we notice that GAN-based training consistently performs better.