Dynamic Data Selection for Neural Machine Translation
Table 4: BLEU scores of data selection using relevance versus random ranking of the bitext. Gradual fine-tuning uses (α=0.5,β=0.7,η=2), with relative training times of 18–20%.
['Ranking', 'Method', 'EMEA', 'Movies', 'TED', 'WMT']
To this end, we repeat all techniques using a bitext which is ranked randomly rather than by its relevance to the test sets. However, the results also show that even in the absence of an appropriate bitext ranking, dynamic data selection—and in particular gradual fine-tuning—is still superior to static data selection. We explain this result as follows: Compared to static selection, both sampling and gradual fine-tuning have better coverage due to their improved exploration of the data. However, sampling also suffers from a surprise effect of observing new data in every epoch. Gradual fine-tuning on the other hand gradually improves learning on a subset of the selected data, suggesting that repetition across epochs has a positive effect on translation quality.