Why Comparing Single Performance Scores Does Not Allow to Draw Conclusions About Machine Learning Approaches
Table 1: The same BiLSTM-CRF approach was evaluated twice under Evaluation 1. The threshold column depicts the average difference in percentage points F1-score for statistical significance with 0.04
['[BOLD] Task', '[BOLD] Threshold  [ITALIC] τ', '[BOLD] % significant', 'Δ( [ITALIC] test)95', 'Δ( [ITALIC] test) [ITALIC] Max']
For the ACE 2005 - Events task, we observe in 34.48% of the cases a significant difference between the models A(j)i and ~A(j)i. For the other tasks, we observe similar results and between 10.72% and 33.20% of the cases are statistically significant.