Machine Translation System Selection from Bandit Feedback
Table 3: Performance on randomly shuffled data. All bandit systems are able to adapt to new domains quickly enough to achieve performance comparable to choosing the single best system, but the contextual bandit significantly outperforms it.
['[EMPTY]', 'R', 'B', 'T']
Here we find that as the data is increasingly mixed, the contextual bandit, LinUCB, significantly outperforms the single-best system. When data is completely shuffled, this amounts to a gain of more than 12 BLEU over the singel best system, a relative improvement of over 37%. This is also true of simpler bandits, and epsilon-greedy also outperforms the single-best system by a narrow margin.