WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipedia
Table 2: Comparison of NMT systems trained on the Europarl corpus and on bitexts automatically mined in Wikipedia by our approach at a threshold of 1.04. We give the number of sentences (first line) and the BLEU score (second line of each bloc) on newstest2014.
['[BOLD] Bitexts', '[BOLD] de-en', '[BOLD] de-fr', '[BOLD] cs-de', '[BOLD] cs-fr']
This is not possible when mining Wikipedia, in particular when considering many language pairs. In this work, we use an evaluation protocol inspired by the WMT shared task on parallel corpus filtering for low-resource conditions Koehn et al. We choose newstest2014 of the WMT evaluations since it provides an N-way parallel test sets for English, French, German and Czech. We favoured the translation between two morphologically rich languages from different families and considered the following language pairs: German/English, German/French, Czech/German and Czech/French. The size of mined bitexts is in the range of 100k to more than 2M (see . We did not try to optimize the architecture of the NMT system to the size of the bitexts and used the same architecture for all systems: the encoder and decoder are 5-layer transformer models as implemented in fairseq Ott et al. The goal of this study is not to develop the best performing NMT system for the considered languages pairs, but to compare different mining parameters. With the exception of Czech/French, we were able to achieve better BLEU scores with the automatically mined bitexts in Wikipedia than with Europarl of the same size. Adding the mined text to the full Europarl corpus, also leads to further improvements of 1.1 to 3.1 BLEU. We argue that this is a good indicator of the quality of the automatically extracted parallel sentences.