Adversarial Neural Networks for Cross-lingual Sequence Tagging
Table 2: es→fr dependency parsing results.
['Training data', 'No ADA', 'GR', 'GAN', 'WGAN']
We train the tagger on ES data with and without adversarial loss. Adversarial training gives consistent improvements over the conventionally trained baseline in all settings. It also outperforms a monolingual model trained on the full dataset of the target language FR which achieves a score of 80.21. When comparing the different adversarial loss functions, GR outperforms GAN and WGAN in most cases. One possible reason is a difference in the discriminator’s strength: During training, we observe that the discriminator of GAN and WGAN could easily predict the language id correctly (even after careful tuning of the update rate between generator and discriminator). This is a well-known problem with training GANs: When the discriminator becomes too strong it provides no useful signal for the feature generator. In contrast, GR training updates the feature generator by taking the inverse of the gradient of the discriminator cross-entropy loss. This simpler setup possibly results in a better training signal for the generator.