Analysis of Automatic Annotation Suggestions for Hard Discourse-Level Tasks in Expert Domains
Table 5: Average annotation time per text (in minutes) and speed-up of our compared to previous annotations.
['[EMPTY]', 'Phase', 'Suggestion A1', 'Suggestion A2', 'Suggestion A3', 'Standard A4', 'Standard A5']
We note that annotators in the Suggestion group (A1–A3) always speed up compared to previous annotations, whereas some of the annotators in the Standard group (A4–A5) slow down. Furthermore, on average, annotators in the Suggestion group exhibit a higher speed-up of annotation time: A1–A3 have a speed-up of 35 % compared to only 21 % for A4–A5 in MeD, and 20 % compared to only 11 % in TeD. Thus, suggestions make the annotation of epistemic activities more efficient.