Story Realization: Expanding Plot Events into Sentences
Table 3: Test set perplexity, BLEU, & ROUGE (F1) scores, with average sentence lengths for event-to-sentence models.
['[BOLD] Model', '[BOLD] Perplexity', '[BOLD] BLEU', '[BOLD] ROUGE', '[BOLD] Length']
Note that some of the models, such as the sentence templates, make no use of gold standard sentences and are thus poorly optimized to use perplexity, BLEU, and ROUGE scores. In addition to running each model in the ensemble individually, we experiment with multiple combinations of the models to assess which combination makes the most effective ensemble. The full ensemble performs better than any of the individual models with regard to perplexity, as it is designed to combine the models such that each of their weaknesses is minimized. The average sentence length metric highlights the differences between the models, with the templates producing the shortest sentences and the finite state machine taking longer to generate sentences due to the constraints it needs to satisfy.