Generating Diverse Story Continuations with Controllable Semantics
Table 10: Human preferences when given three continuations from each pair of systems.
['system 1', 'system 2', '[BOLD] Human Preference 1', '[BOLD] Human Preference 2', '[BOLD] Human Preference neither']
We observe that workers prefer the BS baseline over TS, although TS yields higher diversity. This could be because the continuations from BS are shorter, simpler, and more fluent. In addition, we observe that workers prefer the outputs from the reranking system over BS more often than not. Although the predicted frame system yields more diverse outputs, workers still prefer BS, likely due to the difficulty in predicting frames. The reranking and predicted frame systems are both preferred to TS, though the gap is smaller with the predicted system. We also see that generating helpful suggestions is a difficult task: in many cases workers thought neither system was helpful, especially when given the outputs from BS/TS or TS/predicted.