Towards one-shot learning for rare-word translation with external experts
Table 2: The results of English - Spanish on various domains: TEDTalks and Business. We use AN for using annotations from the phrase table, RF for using REINFORCE (α= 0.5) and CP for using the Copy mechanism.
['System—Data', '[BOLD] dev2010  [BOLD] BLEU', '[BOLD] dev2010  [BOLD] SAC', '[BOLD] dev2010  [BOLD] SUG', '[BOLD] tst2010  [BOLD] BLEU', '[BOLD] tst2010  [BOLD] SAC', '[BOLD] tst2010  [BOLD] SUG', '[BOLD] BusinessTest  [BOLD] BLEU', '[BOLD] BusinessTest  [BOLD] SAC', '[BOLD] BusinessTest  [BOLD] SUG']
First, the main difference between the settings is the SUG and SAC figures for all test sets. Both of them increase dramatically from baseline to annotation, and also increase according to the level of supervision in our model proposals. While the copy mechanism can help us to copy more from the annotation, the REINFORCE models are successfully trained to make the model copy more consistently. Their combination helps us achieve the desired behavior, in which almost all of the annotations given are copied, and we achieve 100% accuracy on the rare-words section that the phrase table covers. As mentioned in the discussion above, the SAC and SUG figures, while being not enough to quantitatively prove that the total number of rare words translated, show that the phrase table is complementary to the neural machine translation, and the more coverage the expert has, the more benefit this method can bring.