Coreferential Reasoning Learningfor Language Representation
Table 5: Results on coreference resolution test sets. Performance on GAP are measured by F1, while scores on the others are given in accuracy. WG: Winogender.
['Model', 'GAP', 'DPR', 'WSC', 'WG', 'PDP']
Our CorefBERT model significantly outperforms BERT, which demonstrates that the intrinsic coreference resolution ability of CorefBERT has been enhanced by involving the mention reference prediction training task. Moreover, it achieves comparable performance with state-of-the-art baseline WikiCREM. Note that, WikiCREM is specially designed for sentence-level coreference resolution and not suitable for other NLP tasks. The capability of CorefBERT in terms of coreferential reasoning can be transferred to other NLP tasks.