Learning from Chunk-based Feedback in Neural Machine Translation
Table 2: Impact of user errors on the translation performance. Under selection ratio% indicates on average what percentage of words in a correct chunk have not been selected in user simulation, but all selected words are correct. Incorrect selection ratio% indicates what percentage of words are incorrectly selected, here the total number of marked words is the same as in chunk-level feedback. In the last row, 10% of marked words are actually incorrect and the total number of marked words is 25% less compared to system in row 1.
['#', '[EMPTY]', 'De-En BLEU', 'De-En TER', 'En-Es BLEU', 'En-Es TER']
We see that randomly dropping 25% of the selection has very little effect on the model, while dropping 50% and more decreases the translation performance significantly, yet still performing at the same level or better than self-training system.