Informative Visual Storytelling with Cross-modal Rules
Table 2. Automatics metric evaluation. Approaches with * are evaluated in photo stream level, while the others are evaluated in album level. Thus scores of approaches with * are provided only for reference.
['Evaluation', 'Method', 'BLUE1-1', 'BLUE-4', 'METEOR-v2', 'ROUGE', 'CIDEr']
Quantitative analysis (VSCMR) model achieves the state-of-the-art performance on most of the metrics. We observe perfomance gain on all of the metrics compared to our baselines: 2.1%, 3.6%, 1.4%, 2%, 3.4% for BLUE-1, BLUE-4, METEOR, ROUGE and CIDEr respectively. This indicates the effectiveness of the inferred semantic concepts. We also observe a CIDEr score gap among different papers and investigate into it. It takes all stories about the same album as references though their photo streams are not always the same. It only takes stories of the same photo stream as references strictly, which will lead to higher CIDEr score and lower score on other metrics. The difference of the evaluation methods lead to variant score distribution.