LAVA NAT: A Non-Autoregressive Translation Model with Look-Around Decoding and Vocabulary Attention
Table 3: The effect of vocabulary attention(VA).
['[BOLD] Method', '[BOLD] WMT14 En→De', '[BOLD] IWSLT14 De→En']
S6SS0SSS0Px2 Effect of Vocabulary Attention(VA) We conduct experiments on our LAVA NAT model on WMT14 En→De and IWSLT14 De→En to show the effectiveness of the LITA mechanism. It demonstrates that incorporating VA can help improve the translating performance (+1.1 on WMT14 En→De and +1.56 on IWSLT14 De→En). This is in line with our expectation that aggregating layer-wise token information in intermediate layers can help improve the decoder’s ability of capturing token-token dependencies .