Dual Conditional Cross-Entropy Filtering of Noisy Parallel Corpora
Table 2: Results on development data. We only train neural models for the 100M sub-task. We did not optimize for any of the other three sub-tasks.
['Filter', 'test16', 'test17']
This model is only used for the computation of reference BLEU scores. The random scores – uniformly sampled values between 0 and 1 – were used to select representative data consisting of 100M words from unprocessed Paracrawl while using the threshold-based selection tool provided by the shared task organizers. The Paracrawl-trained systems (random) has dramatically worse BLEU scores than the WMT18-trained system. Upon manual inspection, we see many untranslated and partially copied sentences in the case of the randomly-selected Paracrawl system. We combined the de-en(x,y) filter with the random scores and trained a corresponding system (de-en⋅random). When reviewing the translated development sets, we did not see any copied/untranslated sentences in the output. We used the domain filter only in combination with the previously introduced filters. This is expected as we optimized for WMT news development sets. We experimented with three cut-off values: 0.00 (no cut-off), 0.25 and 0.50, reaching the highest BLEU scores for a cut-off value c=0.25. We ran two experiments with the similarity based scores, evaluating configurations de-en⋅sim and de-en⋅adq⋅sim⋅dom0.25. The first one corresponds to de-en⋅adq and we compare the effectivness of the adq and sim filters after the application of the language-id-based filter de-en. Interestingly, even when combined with our best scoring scheme (de-en⋅adq⋅dom0.25) resulting in de-en⋅adq⋅sim⋅dom0.25 we see a slight degradation. Based on these results, we do not use the similarity scores for our submission.