Gendered Ambiguous Pronouns Shared Task: Boosting Model Confidence by Evidence Pooling
Table 3: Single model performance on gap-test set by gender. M: masculine, F: feminine, B: (bias) ratio of feminine to masculine performance, O: overall. Log loss is not available for systems that only produce labels. †As reported by Webster et al. Webster et al. (2018). ‡As reported by Liu et al. Liu et al. (2019), their model does not use gold-two-mention labeled span information for prediction.
['[EMPTY]', 'F1 M', 'F1 F', 'F1 B', 'F1 O', 'logloss']
The GREP model (with bert-large-uncased as the language model) achieves a powerful state-of-the-art performance on this task. The model significantly benefits from evidence pooling, gaining 6 points in terms of log loss and 2.8 points in F1 accuracy.