On the Robustness of Unsupervised and Semi-supervised Cross-lingual Word Embedding Learning
Table 4: Accuracy in the cross-lingual natural language inference task (XNLI) using different cross-lingual word embedding models.
['[BOLD] Wikipedia  [BOLD] Sup.', '[BOLD] Wikipedia  [BOLD] Model', '[BOLD] Wikipedia  [BOLD] EN-ES', '[BOLD] Wikipedia  [BOLD] EN-DE', '[BOLD] Wikipedia  [BOLD] EN-RU', '[BOLD] Wikipedia  [BOLD] Avg']
The task of natural language inference (NLI) consists in detecting entailment, contradiction and neutral relations between pairs of sentences. We test a zero-shot cross-lingual transfer setting where a system is trained with English corpora and is then evaluated on a different language. It is important to highlight that in this evaluation our main aim is to compare the quality of the cross-lingual word embeddings, and not to develop a state-of-the-art NLI system. Therefore, since this is a downstream task evaluated at the sentence level (and not at the word level as in dictionary induction and semantic word similarity), we develop a simple bag-of-words approach where a sentence embedding is obtained by word vector averaging. We use the full MultiNLI [williams2018broad] English corpus for training and the Spanish, German and Russian test sets from XNLI [conneau2018xnli] for testing.