Reinforced Video Captioning with Entailment Rewards
Table 2: Our primary video captioning results on MSR-VTT. All CIDEr-RL results are statistically significant over the baseline XE results, and all CIDEnt-RL results are stat. signif. over the CIDEr-RL results. Human* refers to the ‘pairwise’ comparison of human relevance evaluation between CIDEr-RL and CIDEnt-RL models (see full human evaluations of the 3 models in Table 3 and Table 4).
['Models', 'BLEU-4', 'METEOR', 'ROUGE-L', 'CIDEr-D', 'CIDEnt', 'Human*']
First, our baseline attention model trained on cross entropy loss (‘Baseline-XE’) achieves strong results w.r.t. the previous state-of-the-art methods. Next, our policy gradient based mixed-loss RL model with reward as CIDEr (‘CIDEr-RL’) It also achieves statistically significant improvements in terms of human relevance evaluation (see below). Overall, we are also the new Rank1 on the MSR-VTT leaderboard, based on their ranking criteria.