Gendered Ambiguous Pronouns Shared Task: Boosting Model Confidence by Evidence Pooling
Table 8: Example 1 - A comparison of probabilities assigned by ProBERT and GREP
['[EMPTY]', 'P(A)', 'P(B)', 'P(NEITHER)']
While the example is trivial for a human to resolve, a machine would require knowledge of the world to understand “death” and its implications. ProBERT is unsure about the resolution and ends up assigning comparable probabilities to both entities A and B. GREP, on the other hand, is able to shift nearly all the probability mass from B to the correct resolution A, in light of strong evidence presented by the coreference solvers. The above observations indicate that by pooling evidence from various sources, the model is able to reason over a larger space and build a rudimentary form of world knowledge.