A Simple Approach to LearningUnsupervised Multilingual Embeddings
Table 3: Precision@1 for BLI obtained by multilingual algorithms on a highly diverse group of six languages (Arabic, German, English, French, Hindi, and Russian) from the MUSE dataset. The ‘avg’ column reports the average performance of each method. Both the proposed methods, UMML-GW and UMML-SL, obtain better results than UMH and UMWE in this challenging BLI setup, with UMML-SL outperforming every method.
['BilingUnsup', 'ar-de  [BOLD] 46.5', 'ar-en 46.4', 'ar-fr 55.0', 'ar-hi 36.9', 'ar-ru  [BOLD] 35.2', 'de-en 70.8', 'de-fr 61.9', 'de-hi 31.8', 'de-ru 43.6', 'en-fr 79.8', 'en-hi 31.3', 'en-ru 44.1', 'fr-hi 36.1', 'fr-ru 44.9', 'hi-ru 24.9', '[EMPTY]']
Experiment 3: We next aim to learn the multilingual space for a more diverse group of languages from the MUSE dataset: Arabic, German, English, French, Hindi, and Russian. We observe that, except the proposed UMML-SL, all other multilingual methods fail to learn a reasonably good shared multilingual space for all languages. The proposed UMML-GW fails to obtain a reasonable BLI score (<1 Precision@1) in 10 out of 30 pairs. UMWE and UMH suffer from such failure on 16 and 18 pairs, respectively. On the other hand, the proposed UMML-SL obtains effective alignment on all pairs of languages, illustrating its robustness in such challenging setting. It obtains better performance than BilingUnsup, benefiting from transfer learning in such a diverse group as well. However, in its second stage, this “misalignment” does not adversely affect the BLI performance of language pairs not involving Hindi. This can be concluded from the observation that its BLI score for all language pairs not involving Hindi is similar to the corresponding scores obtained by UMML-SL. Overall, UMML-GW is able to learn suitable multilingual embeddings for five languages (ar, de, en, fr, and ru). However, UMH also fails to learn suitable Arabic embeddings in the shared multilingual space. This is surprising since the GW algorithm (employed in the first stage of UMML-GW) learns an effective alignment of English and Arabic words. Hence, it appears that jointly learning the unsupervised alignment and multilingual mapping can adversely affect distant languages (Arabic in this case). Overall, UMH learns suitable multilingual embeddings for four languages (de, en, fr, and ru). ∙ The GAN-based approach, UMWE, learns two groups of alignment in the shared multilingual space. The first group consists of Arabic, English, French, and Hindi language embeddings, which are suitably aligned with each other. However, these are “misaligned” with German and Russian language embeddings in the shared space. On the other hand, the German and Russian language embeddings are suitably aligned with each other (but not with any other language).