WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipedia
Table 6: BLEU scores on the TED test set as proposed in Qi et al. (2018). NMT systems were trained on bitexts mined in Wikipedia only. No other resources were used.
['Lang', 'xx → en', 'en → xx']
We evaluate the trained translation systems on the TED dataset Qi et al. The TED data consists of parallel TED talk transcripts in multiple languages, and it provides development and test sets for 50 languages. Since the development and test sets were already tokenized, we first detokenize them using Moses. We trained NMT systems for all possible language pairs with more than twenty-five thousand mined sentences. This gives us in total 1886 language pairs in 45 languages. We train L1→L2 and L2→L1 with the same mined bitexts L1/L2. Due to space constraints, we are unable to report BLEU score for all language combinations in that table. 23 NMT systems achieve BLEU scores over 30, the best one being 37.3 for Brazilian Portuguese to English. Several results are worth mentioning, like Farsi/English: 16.7, Hebrew/English: 25.7, Indonesian/English: 24.9 or English/Hindi: 25.7 We also achieve interesting results for translation between various non English language pairs for which it is usually not easy to find parallel data, e.g. Norwegian ↔ Danish ≈33, Norwegian ↔ Swedish ≈25, Indonesian ↔ Vietnamese ≈16 or Japanese / Korean ≈17.