Detecting Everyday Scenarios in Narrative Texts
Table 3: Relative agreement on segment spans between annotated segments that overlap by at least one token and are assigned the same scenario label
['[BOLD] Annotators', '2', '3', '4']
We used fuzzy matching to calculate agreement in span between segments that overlap by at least one token. On average, the annotators achieve 67% agreement on segment spans. This shows considerable segment overlap when both annotators agreed that a particular scenario is referenced.