Gendered Ambiguous Pronouns Shared Task: Boosting Model Confidence by Evidence Pooling
Table 5: Class-wise comparison of model accuracy for ProBERT and GREP. Off-diagonal terms show cases where GREP fixes errors made by ProBERT and vice-versa.
['[EMPTY]', 'ProBERT', 'GREP Incorrect', 'GREP Correct']
It can be seen that GREP is more confident in its predictions (all distributions appear translated closer to 1.0), and the improvement is overwhelmingly evident for the NEITHER class. The diagonal terms show the number of instances that the two models agree on, and the off-diagonal terms show where they disagree. The numbers reveal that the evidence pooling module not only boosts the model confidence but also helps in correctly resolving Neither instances (44 vs 11), indicating that the model is successfully able to build evidence for or against the given candidates.