The Effect of Different Writing Tasks on Linguistic Style:A Case Study of the ROC Story Cloze Task
Table 2: Results of experiments 1 (right vs. wrong), 2 (original vs. right (new)) and 3 (original vs. wrong (new) endings). In all cases, our setup implies a 50% random baseline.
['[BOLD] Experiment', '[BOLD] Accuracy']
In all experiments, our model achieves performance well above what would be expected under chance (50% by design). Noting again that our model ignores the story context (the preceding four sentences), our model is unable to capture any notion of coherence. This finding provides strong evidence that the authors’ style was affected by the writing task they were given to perform.