Knowledge-Enriched Visual Storytelling
Table 3: Direct comparison evaluation without given photos. Numbers indicate average rank given to stories (from 1 to 5, lower is better).
['Human Evaluation (Story Displayed  [BOLD] without Images)', 'Human Evaluation (Story Displayed  [BOLD] without Images)  [BOLD] GLAC ', 'Human Evaluation (Story Displayed  [BOLD] without Images)  [BOLD] No KG', 'Human Evaluation (Story Displayed  [BOLD] without Images)  [BOLD] OpenIE', 'Human Evaluation (Story Displayed  [BOLD] without Images)  [BOLD] Visual Genome', 'Human Evaluation (Story Displayed  [BOLD] without Images)  [BOLD] Human']
We also explored the relation between images and the generated stories in visual storytelling. To determine whether the generated stories are well matched to the photos, we removed the photos and conducted another overall story comparison. The ranking of visual stories from the state-of-the-art model remains the same. This suggests that visual stories from KG-Story better fit the photos, where the images and texts support each other. The visual story from the state-of-the-art GLAC clearly is not grounded: there is no snow in the second photo, no kids in the fourth, and no baby in the fifth, though only considering the text it is a reasonable Christmas story. In contrast, given the photos, the proposed KG-Story generates an obviously better story. This example also illustrates that the given images are both hints and constraints in visual storytelling, which makes it a more challenging task than general story generation. Besides, even when photos are not provided, the average rank of visual stories from KG-Story is comparable to that from the state-of-the-art model. This demonstrates that our stories are both closely aligned to the images and also stand without the images, which we attribute to the use of the additional Virtual Genome dataset and ROCStories Corpora for the model learning.