Data Selection with Cluster-Based Language Difference Modelsand Cynical Selection
Table 4: Perplexity scores on the TED DE (input) corpus for the models trained with 2M and 6M selected sentences.
['Method', 'ppl at 2M', 'ppl at 6M']
Each of the cluster-based data selection methods on the German side outperformed their vanilla Moore-Lewis counterparts (comparing monolingual cluster-based vs monolingual Moore-Lewis, both on the German side, and comparing bilingual cluster-based vs bilingual Moore-Lewis). At 6M sentences selected, near convergence, the cluster-based methods are each 20 perplexity points better than the standard cross-entropy difference, but the cynical selection method is slightly better.