Self-Regulated Interactive Sequence-to-Sequence Learning
Table 3: Evaluation of models at early stopping points. Results for three random seeds on IWSLT are averaged, reporting the standard deviation in the subscript. The translation of the dev set is obtained by greedy decoding (as during validation) and of the test set with beam search of width five. The costs are measured in character edits and clicks, as described in Section 4.
['[BOLD] Model', '[BOLD] IWSLT dev  [BOLD] BLEU↑', '[BOLD] IWSLT dev  [BOLD] Cost↓', '[BOLD] IWSLT test  [BOLD] BLEU↑', '[BOLD] IWSLT test  [BOLD] TER↓']
All models notably improve over the baseline, only using full feedback leads to the overall best model on IWSLT (+0.6 BLEU / -0.6 TER), but costs a massive amounts of edits (417k characters). Self-regulating models still achieve improvements of 0.4–0.5 BLEU/TER with costs reduced up to a factor of 23. The reduction in cost is enabled by the use of cheaper feedback, here markings and self-supervision, which in isolation are successful as well. Self-supervision works surprisingly well, which makes it attractive for cheap but effective unsupervised domain adaptation. It has to be noted that both weak and self-supervision worked only well when targets were pre-computed with the baseline model and held fixed during training. We suspect that the strong reward signal (ft=1) for non-reference outputs leads otherwise to undesired local overfitting effects that a learner with online-generated targets cannot recover from.