CIDEr: Consensus-based Image Description Evaluation
Table 1: Results on four kinds of pairs for PASCAL-50S and two kinds of pairs for ABSTRACT-50S. The best performing method is shown in bold. Note: we use ROUGEL for PASCAL-50S and ROUGE1 for ABSTRACT-50S
['Metric', 'PASCAL-50S HC', 'PASCAL-50S HI', 'PASCAL-50S HM', 'PASCAL-50S MM', 'ABSTRACT-50S HC', 'ABSTRACT-50S HI']
We next show the best performing versions of the metrics CIDEr, BLEU, ROUGE and METEOR on PASCAL-50S and ABSTRACT-50S, respectively, for different kinds of candidate pairs As discussed in Sec. HC, (human–human incorrect) HI, (human–machine) HM, and (machine–machine) MM. We find that out of six cases, our proposed automated metric is best in five. We show significant gains on the challenging MM and HC tasks that involve differentiating between fine-grained differences between sentences (two machine generated sentences and two human generated sentences). This result is encouraging because it indicates that the CIDEr metric will continue to perform well as image description methods continue to improve. On the easier tasks of judging consensus on HI and HM pairs, all methods perform well.