PlotMachines: Outline-Conditioned Generation with Dynamic Plot State Tracking
Table 2: ROUGE Results on Wiki, WritingPrompts and NYTimes Datasets. The top block represents the baseline models on story/article generation, while the bottom blocks include ablations of our PlotMachines models.
['Model', '[BOLD] Wikiplots R-1', '[BOLD] Wikiplots R-2', '[BOLD] Wikiplots R-L', '[BOLD] WritingPrompts R-1', '[BOLD] WritingPrompts R-2', '[BOLD] WritingPrompts R-L', '[BOLD] New York Times R-1', '[BOLD] New York Times R-2', '[BOLD] New York Times R-L']
Ablated PlotMachines Models First, we use the base GPT and GPT2 models, that are fine-tuned similarly to our model but using only outline inputs (without memory, preceding context, or discourse representations). Second, we investigate the effects of using the preceding context representation but still excluding memory and discourse tokens (PM-NoMem-NoDisc). Lastly, we use PM-NoMem, a model variant that excludes the memory but still uses outline, discourse, and preceding context representations as input. Results show that the full PlotMachines achieves comparable or higher ROUGE scores on all three datasets. Both PlotMachines variants (using GPT or GPT-2 as a base) achieve significant improvement over Grover (even though Grover includes more layers and parameters than the model using GPT).