Classical Structured Prediction Losses for Sequence to Sequence Learning
Table 2: Validation and test BLEU for loss combination strategies. We either use token-level TokLS and sequence-level Riskindividually or combine them as a weighted combination, a constrained combination, a random choice for each sample, cf. §3.3.
['[EMPTY]', '[BOLD] valid', '[BOLD] test']
Next, we compare various strategies to combine sequence-level and token-level objectives (cf. For these experiments we use 5 candidate sequences per training example for faster experimental turnaround. We consider Risk as sequence-level loss and label smoothing as token-level loss. We also compare to randomly choosing between token-level and sequence-level updates and find it underperforms the more principled constrained strategy. In the remaining experiments we use the weighted strategy.