Evaluating Text-to-Image Matching using Binary Image Selection (BISON)
Table 6: Performance of three image captioning systems in terms of four captioning scores on the COCO validation set (left) and in terms of BISON accuracy on the COCO-BISON dataset (right). Human performances marked with ⋆ were measured on the COCO test set. See text for details.
['Dataset → Measure →', 'COCO validation split  [BOLD] BLEU-4', 'COCO validation split  [BOLD] CIDEr', 'COCO validation split  [BOLD] SPICE', 'COCO validation split  [BOLD] METEOR', 'COCO-BISON  [BOLD] BISON']
Again, the results reveal that the ranking of the three systems is identical across all evaluation scores, even though BISON measures different aspects of the system than the captioning scores.