Argumentation Mining in User-Generated Web Discourse
Table 8: Class distribution of the gold data Toulmin corpus approximated to the sentence level boundaries.
['[BOLD] Class', '[BOLD] Sentences in data  [BOLD] Relative (%)', '[BOLD] Sentences in data  [BOLD] Absolute', '[BOLD] Class', '[BOLD] Sentences in data  [BOLD] Relative (%)', '[BOLD] Sentences in data  [BOLD] Absolute']
The little presence of rebuttal and refutation (4 classes account only for 3.4% of the data) makes this dataset very unbalanced. The overall best performance (Macro-F1 0.251) was achieved using the rich feature sets (01234 and 234) and significantly outperformed the baseline as well as other feature sets. Classification of non-argumentative text (the "O" class) yields about 0.7 F1 score even in the baseline setting. The boundaries of claims (Cla-B), premises (Pre-B), and backing (Bac-B) reach in average lower scores then their respective inside tags (Cla-I, Pre-I, Bac-I). It can be interpreted such that the system is able to classify that a certain sentence belongs to a certain argument component, but the distinction whether it is a beginning of the argument component is harder. The very low numbers for rebuttal and refutation have two reasons.