EMNLP’18 Zero-Shot Open Entity Typing as Type-Compatible Grounding
Table 3: Evaluation of coarse entity-typing (§4.2): we compare two supervised entity-typers with our system. For the supervised systems, cells with gray color indicate in-domain evaluation. For each column, the best, out-of-domain and overall results are bold-faced and underlined, respectively. Numbers are F1 in percentage. In most of the out-of-domain settings our system outperforms the supervised system.
['System', 'Trained on', 'OntoNotes PER', 'OntoNotes LOC', 'OntoNotes ORG', 'CoNLL PER', 'CoNLL LOC', 'CoNLL ORG', 'MUC PER', 'MUC LOC', 'MUC ORG']
We focus on three types that are shared among the datasets: PER, LOC, ORG. In coarse-entity typing, the best available systems are heavily supervised. In this evaluation, we use gold mention spans; i.e., we force the decoding algorithm of the supervised systems to select the best of the three classes for each gold mention. As expected, the supervised systems have strong in-domain performance. However, they suffer a significant drop when evaluated in a different domain. Our system, while not trained on any supervised data, achieves better or comparable performance compared to other supervised baselines in the out-of-domain evaluations.