Exploiting stance hierarchies for cost-sensitivestance detection of Web documents
Table 5. Document stance classification performance.
['System', 'FNC', 'F1 [ITALIC] m', 'F1Unrel.', 'F1Neutral', 'F1Agree', 'F1Disagr.', 'F1 [ITALIC] mAgr/Dis']
First, we notice that, considering the problematic FNC-I evaluation measure, CombNSE achieves the highest performance (0.83), followed by SOLAT, Athene, UCLMR, StackLSTM, 3-Stage Trad. (0.82) and our pipeline approach (0.81). However, we see that the ranking of the top performing systems is very different if we consider the more robust macro-averaged F1 measure (F1m). Specifically, L3S now achieves the highest score (0.62), outperforming the best baseline system (stackLSTM) by one percentage point, while CombNSE is now in the fourth position (F1m = 0.59).