CompRes: A Dataset for Narrative Structure in News
Table 1: Overview of the CompRes dataset. Note that the categories are not mutually exclusive, due to the multi-labeled nature of the annotation scheme
['[EMPTY]', '[BOLD] Complication', '[BOLD] Resolution', '[BOLD] Success']
Here we describe the process of constructing CompRes, our dataset of news articles annotated with narrative structures. The dataset contains 29 news articles, comprising 1,099 sentences. The categories are unevenly distributed: Complication is significantly more frequent than Resolution and Success. This was to be expected, considering the known biases of ”newsworthiness” towards problems, crises and scandals, and due to the fact that in news media, resolutions often follow reported complications. This is most evident in the baseline models – F1 scores of 0.61, 0.4 and 0.15 in the SVM model, and F1 scores of 0.61, 0.43 and 0.17 in the augmented SVM model for Complication, Resolution and Success (respectively). However, in the deep models this behavior seems to be less extreme; in the augmented RoBERTa model, the F1 score for the Success category is higher by 0.05 compared to the Resolution category, despite being less frequent in the dataset. We also observe that the Success category consistently exhibit notably higher precision than recall, across all models, possibly due to the smaller number of samples encountered by the classifier during training. This is generally true for the Resolution category as well (except in the case of the RoBERTa model), though to a lesser extent.