Keep it Consistent: Topic-Aware Storytelling from an Image Stream via Iterative Multi-agent Communication
Table 4: Human evaluation results on pairwise comparison.
['Choice (%)', 'TAVST  [ITALIC] vs VST TAVST', 'TAVST  [ITALIC] vs VST VST', 'TAVST  [ITALIC] vs VST Tie', 'TAVST  [ITALIC] vs AREL TAVST', 'TAVST  [ITALIC] vs AREL AREL', 'TAVST  [ITALIC] vs AREL Tie', 'TAVST  [ITALIC] vs GT TAVST', 'TAVST  [ITALIC] vs GT GT', 'TAVST  [ITALIC] vs GT Tie']
A good story for an image stream should have three significant factors: (1) Relevance: the story should be relevant to the image stream. (2) Expressiveness: the story should be concrete and coherent, and have a human-like language style. (3) Topic Consistency: the story should be consistent to the topic. We compare our model with three other models in terms of these three metrics: TAVST vs VST, TAVST vs AREL and TAVST vs ground-truth. In this annotation task, AMT evaluators need to compare two given stories according to these three factors and choose which story is better in terms of a certain factor. Our model performs better than the other two models in terms of relevance and topic consistency. The advantage of topic consistency is more promising. This proves that the topic description generator can help the story generation agent construct a more consistent story.