Summarizing Dialogic Arguments from Social Media
Table 3: Sentence distribution in each domain.
['Topic', 'Important', 'Not Important']
We now have one or more labels for each sentence in a dialog, but we are primarily interested in the tier rank of the sentences. We group labels by tier and compute the average tier label per sentence. We define any sentence with an average tier score of 3 or higher as important. Thus, steps S1, S2 and S3 above are simply carried out to arrive at a well-motivated and theoretically grounded definition of important argument, and the task we address in this paper is binary classification applied to dialogs to select sentences that are important. The average Cohen’s kappa between the annotators is respectable, with a kappa value of 0.68 for gun control, 0.63 for abortion, and 0.62 for gay marriage.