Semi-Autoregressive Training Improves Mask-Predict Decoding
Table 4: Increasing the number of forward passes used to produce each training example in SMART can negatively effect the resulting model. Performance measured on WMT’14 DE-EN (development set BLEU).
['[BOLD] Forward Passes', '[BOLD] Decoding Iterations  [BOLD] 1', '[BOLD] Decoding Iterations  [BOLD] 4', '[BOLD] Decoding Iterations  [BOLD] 10']
We first consider creating our training examples by performing multiple mask-predict iterations during training, instead of just two. worse results.