Generating Narrative Text in a Switching Dynamical System
Table 2: F1 scores for ROUGE-1, 2, and L and METEOR (M) (default mode score) for randomly sampled 500 stories from the test set. #Sents(s) column represents the “fill in” sentence(s) that the models generated using Gibbs sampling. Our SLDS models pick the best of 50 samples, the baselines models pick the best of 1000 samples
['# Sent(s)', 'Metric', 'SLDS', 'SLDS-1', 'SLDS-10', 'SLDS-25', 'SLDS-50', 'S2S', 'LM', 'LDS']
using the true sentences as targets. The #Sent(s) column indicates which sentence(s) were removed, and then regenerated by the model. We gave the baselines a slight edge over SLDS because they pick the best out of 1000 samples while SLDS is only out of 50. The SLDS models see their largest gain over the baseline models when at least the first sentence is given as an input. The baseline models do better when the first and second sentence need to be imputed. This is likely due to the fact that having access to the earlier sentences allows a better initialization for the Gibbs sampler. Surprisingly, the semi-supervised variants of the SLDS models achieve higher scores. The reasons for this is discussed below in the Perplexity section.