Modelling Suspense in Short Stories as Uncertainty Reduction over Neural Representation
Table 1: For accuracy the baseline probability is 1 in 99; k=10 is the accuracy of the top 10 sentences of the batch. From the best epoch of training on the WritingPrompts development set.
['[EMPTY]', '[BOLD] GRU', '[BOLD] LSTM']
We used the pretrained GPT weights but fine-tuned the encoder and decoder weights on our task. For the RNN components of our hierarchical model, we experimented with both GRU (chung2015gated) and LSTM (DBLP:journals/neco/HochreiterS97) variants. The GRU model had two layers in both sen_enc and story_enc; the LSTM model had four layers each in sen_enc and story_enc. Both had two fusion layers and the size of the hidden layers for both model variants was 768. Both perform similarly, with slightly worse loss for the LSTM variant, but faster training and better generation accuracy. Overall, model performance is strong: the LSTM variant picks out the correct sentence 54% of the time and generates it 46% of the time. This indicates that our architecture successfully captures the structure of stories.