AMALGUM – A Free, Balanced, Multilayer English Web Corpus
Table 5: Coreference Resolution Performance. The average F1 of MUC, B3 and CEAFϕ4 is the main evaluation metric.
['[EMPTY]', 'MUC P', 'MUC R', 'MUC F1', 'B3 P', 'B3 R', 'B3 F1', 'CEAF [ITALIC] ϕ4 P', 'CEAF [ITALIC] ϕ4 R', 'CEAF [ITALIC] ϕ4 F1', 'Avg. F1']
Since the training data for GUM is limited, \newciteJoshi2019BERTFC’s model does not achieve satisfying scores out-of-the-box on GUM test as in the CoNLL dataset. However in the highest scoring xrenner model, it turns out that entity types for many non-coreferential mentions are more often incorrect than for We therefore considered using that system for mention detection and classification, and report slightly lower scores using this combination of entity recognition and coreference resolution (xrenner + S&H in the table). The modified xrenner significantly outperforms \newciteshibuya_nested_2019 on AMALGUM and only scores 1.78% lower on GUM test.