A Hierarchical Approach for Generating Descriptive Image Paragraphs
Table 2: Main results for generating paragraphs. Our Region-Hierarchical method is compared with six baseline models and human performance along six language metrics.
['[EMPTY]', 'METEOR', 'CIDEr', 'BLEU-1', 'BLEU-2', 'BLEU-3', 'BLEU-4']
We present our main results at generating paragraphs in Tab. The Sentence-Concat method performs poorly, achieving the lowest scores across all metrics. Its lackluster performance provides further evidence of the stark differences between single-sentence captioning and paragraph generation. Surprisingly, the hard-coded template-based approach performs reasonably well, particularly on CIDEr, METEOR, and BLEU-1, where it is competitive with some of the neural approaches. However, its relatively poor performance on BLEU-3 and BLEU-4 highlights the limitation of reasoning about regions in isolation – it is unable to produce much text relating regions to one another, and further suffers from a lack of “connective tissue” that transforms paragraphs from a series of disconnected thoughts into a coherent whole. DenseCap-Concat scores worse than Template on all metrics except CIDEr, illustrating the necessity of Template’s caption parsing and recombination. To make these metrics more interpretable, we performed a human evaluation by collecting an additional paragraph for 500 randomly chosen images, with results in the last row of Tab. As expected, humans produce superior descriptions to any automatic method, performing better on all language metrics considered.