From News to Medical: Cross-domain Discourse Segmentation
Table 5: Average inter-annotator agreement per section, ordered from highest to lowest, the corresponding average F1 of the neural segmenter, and number of tokens (there are 2 documents per section, except 1 for Summary).
['Section', 'Kappa', 'F1', '#tokens']
Here we compare the level of annotator agreement with the performance of the neural segmenter. However, high agreement does not always translate to good performance. The Introduction section is straightforward for the annotators to segment, but this is also where most citations occur, causing the segmenter to perform more poorly. Earlier, we had noted the Discussion section was the hardest for annotators to label because of the more complex discourse. These more ambiguous syntactic constructions also pose a challenge for the segmenter, with lower performance than most other sections.