Back-Translation Sampling by Targeting Difficult Words in Neural Machine Translation
Table 8: German↔English translation quality (Bleu). Experiments marked † are averaged over 3 runs. PredLoss is the contextual prediction loss and MeanPredLoss is the average loss. token and Sword are context selection definitions from neighboring tokens and subword units respectively. Note that token includes both subword units and full words. emb is computing context similarities with token embeddings and match is comparing the context tokens.
['[BOLD] System', '[EMPTY]', '[EMPTY]', '[BOLD] De-En test2014', '[BOLD] De-En test2015', '[BOLD] De-En test2016', '[BOLD] De-En test2017', '[BOLD] En-De test2014', '[BOLD] En-De test2015', '[BOLD] En-De test2016', '[BOLD] En-De test2017']
In these experiments, we set the hyperparameters s and w to 0.75 and 4, respectively. Comparing the experiments with different similarity measures, match and emb, we observe that in all test sets we achieve the best results when using word embeddings. This indicate that for targeted sampling, it is more beneficial to have diversity in the context of difficult words as opposed to having the exact ngrams.