Bandit Structured Prediction for Learning from Partial Feedback in Statistical Machine Translation
Table 1: Corpus BLEU (under MAP decoding) on test set for SMT domain adaptation from Europarl to NewsCommentary by k-best reranking.
['full information  [BOLD] in-domain SMT', 'full information  [BOLD] out-domain SMT', 'bandit information  [BOLD] DuelingBandit', 'bandit information  [BOLD] BanditStruct']
We show the standard evaluation of the corpus-BLEU metric evaluated under MAP inference. The range of possible improvements is given by the difference of the BLEU score of the in-domain model and the BLEU score of the out-of-domain model – nearly 3 BLEU points. Bandit learning can improve the out-of-domain baseline by about 1.26 BLEU points (Bandit Structured Prediction) and by about 1.52 BLEU points (Dueling Bandits). The progression of loss values is smoother and empirical convergence speed is faster for Dueling Bandits since it can exploit preference judgements instead of having to trust real-valued feedback.