HiExpan: Task-Guided Taxonomy Construction by Hierarchical Tree Expansion
Table 3. Quantitative results: we show the quantitative results of the taxonomies constructed by HSetExpan, NoREPEL, NoGTO, and HiExpan. Pa, Ra, F1a denote the ancestor-Precision, ancestor-Recall, and ancestor-F1-score, respectively. Similarly, we denote edge-based metrics as Pe, Re, and F1e, respectively.
['[BOLD] Method', '[BOLD] Wiki  [ITALIC] Pa', '[BOLD] Wiki  [ITALIC] Ra', '[BOLD] Wiki  [ITALIC] F1 [ITALIC] a', '[BOLD] Wiki  [ITALIC] Pe', '[BOLD] Wiki  [ITALIC] Re', '[BOLD] Wiki  [ITALIC] F1 [ITALIC] e', '[BOLD] DBLP  [ITALIC] Pa', '[BOLD] DBLP  [ITALIC] Ra', '[BOLD] DBLP  [ITALIC] F1 [ITALIC] a', '[BOLD] DBLP  [ITALIC] Pe', '[BOLD] DBLP  [ITALIC] Re', '[BOLD] DBLP  [ITALIC] F1 [ITALIC] e', '[BOLD] PubMed-CVD  [ITALIC] Pa', '[BOLD] PubMed-CVD  [ITALIC] Ra', '[BOLD] PubMed-CVD  [ITALIC] F1 [ITALIC] a', '[BOLD] PubMed-CVD  [ITALIC] Pe', '[BOLD] PubMed-CVD  [ITALIC] Re', '[BOLD] PubMed-CVD  [ITALIC] F1 [ITALIC] e']
Then we pool all the edges together and ask five people, including the second and third author of this paper as well as three volunteers, to judge these pairs independently. We show them seed parent-child pairs as well as the generated parent-child pairs, and ask them to evaluate whether the generated parent-child pairs have the same relation as the given seed parent-child pairs. After collecting these answers from the annotators, we simply use majority voting to label the pairs. We then use these annotated data as the gold standard. The labeled dataset is available at: http://bit.ly/2Jbilte.