A Better Variant of Self-Critical Sequence Training
Table 4: Replacing the greedy decoding output c∗ in SCST with a separately drawn sample ^c′.
['[EMPTY]', 'Bleu1', 'Bleu2', 'Bleu3', 'Bleu4', 'ROUGE_L', 'METEOR', 'CIDEr', 'SPICE']
This is as expected, because using one sample to estimate the expected reward is too noisy, resulting in larger gradient variance, while the reward of greedy decoding output may be biased but more stable. It also shows that it is important to use sufficiently large K to have a better estimation of expected reward.