Classical Structured Prediction Losses for Sequence to Sequence Learning
Table 4: Generating candidates online or offline.
['[EMPTY]', '[BOLD] valid', '[BOLD] test']
Next, we consider the question if refreshing the candidate subset at every training step (online) results in better accuracy compared to generating candidates before training and keeping the set static throughout training (offline). However the online setting is much slower, since regenerating the candidate set requires incremental (left to right) inference with our model which is very slow compared to efficient forward/backward over large batches of pre-generated hypothesis.