From Credit Assignment to Entropy Regularization: Two New Algorithms for Neural Sequence Prediction
Table 1: Test results on two benchmark tasks. Bold faces highlight the best in the corresponding category.
['[BOLD] Algorithm', '[BOLD] MT (w/o input feeding) Mean', '[BOLD] MT (w/o input feeding) Min', '[BOLD] MT (w/o input feeding) Max', '[BOLD] MT (w/ input feeding) Mean', '[BOLD] MT (w/ input feeding) Min', '[BOLD] MT (w/ input feeding) Max', '[BOLD] Image Captioning Mean', '[BOLD] Image Captioning Min', '[BOLD] Image Captioning Max']
The results on MT are summarized in the left half of Tab. Firstly, all four advanced algorithms significantly outperform the MLE baseline. More importantly, both VAML and ERAC improve upon their direct baselines, RAML and AC, by a clear margin on average. The result suggests the two proposed algorithms both well combine the benefits of a delicate credit assignment scheme and the entropy regularization, achieving improved performance.