Noisy Parallel Approximate Decoding for Conditional Recurrent Language Model
Table 1: Effect of noise injection. For both stochastic sampling and NPAD, we used 50 parallel samplers. For NPAD, we used the greedy decoding as an inner-decoding strategy.
['Strategy', '[ITALIC] σ0', 'Valid NLL↓', 'Valid BLEU↑', 'Test-1 NLL↓', 'Test-1 BLEU↑']
As expected, the proposed NPAD improves upon the deterministic greedy decoding as well as the stochastic sampling strategy. It is important to notice that the improvement by the NPAD is significantly larger than that by the stochastic sampling, which confirms that it is more efficient and effective to inject noise in the hidden state of the neural network.