LAVA NAT: A Non-Autoregressive Translation Model with Look-Around Decoding and Vocabulary Attention
Table 6: The effect of different sampling mechanisms. “TF”, “SS” and “DSS” are short for “Teacher Forcing”, “Scheduled Sampling” and “Differentiable Scheduled Sampling”, respectively.
['[BOLD] Method', '[BOLD] WMT14 En→De', '[BOLD] IWSLT14 De→En']
We conduct ablation experiments on WMT14 En→DE and IWSLT14 De→EN with the LAVA NAT model. Teacher forcing gets the worst performance while differentiable scheduled sampling the best and scheduled sampling is slightly better than teacher forcing. We believe this is the differentiable nature of DSS that makes the model easy to learn and thus alleviate the “exposure bias” issue.