Adversarial Learning with Contextual Embeddings for Zero-resource Cross-lingual Classification and NER
Table 1: Classification accuracy on the MLDoc test sets. We present results for BERT finetuned on labeled English data and BERT finetuned on labeled English data with language-adversarial training. Our results are averaged across 4 training runs, and hyperparameters are tuned on English dev data.
['[EMPTY]', 'En', 'De', 'Es', 'Fr', 'It', 'Ja', 'Ru', 'Zh']
Generally, adversarial training improves the accuracy across all languages, and the improvement is sometimes dramatic versus the BERT non-adversarial baseline.