Explicit Cross-lingual Pre-training for Unsupervised Machine Translation
Table 4: Results of zero-shot cross-lingual classification (on XNLI test sets).
['Method', 'en', 'fr', 'de']
We also conduct experiments on the cross-lingual classification task Conneau et al. using the cross-lingual language inference (XNLI) dataset Conneau et al. Specifically, we add a linear classification layer on top of the first hidden state of our pre-trained model and fine-tune its parameters on the English NLI dataset. We can find that our method reaches a new record of the zero-shot cross-lingual classification task on languages of French (fr) and Germany (de), which confirms again that our CMLM works better on modeling cross-lingual information than previous methods in the unsupervised scenario.