Vocabulary Selection Strategies for Neural Machine Translation
Table 1: Final decoding accuracy results for WMT English-German and English-Romanian on various test sets (newstest 2010 – 2016, except 2013 our validation set). We report the average vocabulary size per sentence, coverage of the reference and decoding time in milliseconds for newstest2015 and newstest2016. The parameter column indicates the maximum number of selected candidates per source word or phrase.
['[BOLD] EN-DE', '[BOLD] Param.', '[BOLD] 2010', '[BOLD] 2011', '[BOLD] 2012', '[BOLD] 2014', '[BOLD] 2015', '[BOLD] Voc.', '[BOLD] Cov.', '[BOLD] Time']
What is the exact speed and accuracy trade-off when reducing the output vocabulary? For our best methods (word alignments, phrase alignments and SVMs) we pick points such that vocabularies are kept small while maintaining good accuracy compared to the full vocabulary setting. For co-occurrence counts and bilingual PCA we choose settings with comparable speed. On English-German translation we achieve more than a 10-fold speed-up over the full vocabulary setting. Accuracy for the word alignment-based selection matches the full vocabulary setting on most test sets or decreases only slightly. On English-Romanian, we achieve a speed-up of over 5 times with word alignments at 28.1 BLEU versus 27.9 BLEU for the full vocabulary baseline. The smaller speed-up on English-Romanian is due to the smaller vocab of the baseline in this setting which is 50k compared to 100k for English-German. This is because training scores the vocabulary exactly once per target position, while beam search has to score multiple hypotheses at each generation step.