DiscoFuse: A Large-Scale Dataset for Discourse-Based Sentence Fusion
Table 5: Average agreement for each fusion decision between the gold annotation and rater majority on examples marked as understandable by the raters. The right column considers only examples in which both the ‘gold’ and rater majority agreed that a connective should be added.
['Decision', 'All Examples (%)', 'Examples with']
We further looked at the agreement between the rater majority and the ‘gold’ fusion decision. Consensus on whether to add an anaphor is higher, but not very high (63%), especially in sentences when the anaphor in t is a nominal rather than a pronoun. Finally, there is higher agreement on selecting the connective category (57%), for which the random baseline is 20%.