No Metrics Are Perfect: Adversarial Reward Learning for Visual Storytelling
Table 4: Pairwise human comparisons. The results indicate the consistent superiority of our AREL model in generating more human-like stories than the SOTA methods.
['Choice (%)', 'AREL  [ITALIC] vs XE-ss AREL', 'AREL  [ITALIC] vs XE-ss XE-ss', 'AREL  [ITALIC] vs XE-ss Tie', 'AREL  [ITALIC] vs BLEU-RL AREL', 'AREL  [ITALIC] vs BLEU-RL BLEU-RL', 'AREL  [ITALIC] vs BLEU-RL Tie', 'AREL  [ITALIC] vs CIDEr-RL AREL', 'AREL  [ITALIC] vs CIDEr-RL CIDEr-RL', 'AREL  [ITALIC] vs CIDEr-RL Tie', 'AREL  [ITALIC] vs GAN AREL', 'AREL  [ITALIC] vs GAN GAN', 'AREL  [ITALIC] vs GAN Tie']
S4SS3SSS0Px2 Pairwise Comparison In order to have a clear comparison with competing algorithms with respect to different semantic features of the stories, we further perform four pairwise comparison tests: AREL vs XE-ss/BLEU-RL/CIDEr-RL/GAN. For each photostream, the worker is presented with two generated stories and asked to make decisions from the three aspects: