A Massive Collection of Cross-Lingual Web-Document Pairs
Table 3: BLEU scores of NMT models trained on bitext data mined from aligned documents on TED Talk test sets. Volume given as number of distinct aligned sentence pairs.
['[BOLD] Language', '[BOLD] BLEU En– [ITALIC] x', '[BOLD] BLEU  [ITALIC] x–En', '[BOLD] Vol']
the number of distinct aligned sentence pairs (reported in millions). Based on these results, it appears that European languages yield higher quality sentences than non-European regardless of the resource level of the direction. Additionally, documents aligned across high-resource directions yield enough high-quality aligned data to learn high-quality models. While these bleu scores should be taken in context of the volume of aligned bitexts, one can get an intuition as to the quality of the underlying URL-aligned documents the sentences were mined from from the resultant test-set BLEU scores.