Argumentation Mining in User-Generated Web Discourse
Table 11: Results of classification of argument components in the cross-domain scenario. Macro-F1 scores reported, bold numbers denote the best results. HS – homeschooling, MS – mainstreaming, PIS – prayer in schools, PPS – private vs. public schools, RS – redshirting, SSE – single sex education. Results in the aggregated row are computed from an aggregated confusion matrix over all domains. The differences between the best feature set combination (4) and others are statistically significant (p<0.001; paired exact Liddell’s test).
['Domain', '[BOLD] Feature set combinations 0', '[BOLD] Feature set combinations 01', '[BOLD] Feature set combinations 012', '[BOLD] Feature set combinations 0123', '[BOLD] Feature set combinations 01234', '[BOLD] Feature set combinations 1234', '[BOLD] Feature set combinations 234', '[BOLD] Feature set combinations 34', '[BOLD] Feature set combinations 4']
However, using only feature set 4 (embeddings), the system performance increases rapidly, so it is even comparable to numbers achieved in the in-domain scenario. These results indicate that embedding features generalize well across domains in our task of argument component identification.