Improving Distantly-supervised Entity Typing with Compact Latent Space Clustering
Table 1: Performance comparision of FET systems on the two datasets.
['[BOLD] Method', '[BOLD] Method', '[BOLD] OntoNotes  [BOLD] Strict Acc.', '[BOLD] OntoNotes  [BOLD] Macro F1', '[BOLD] OntoNotes  [BOLD] Micro F1', '[BOLD] BBN  [BOLD] Strict Acc.', '[BOLD] BBN  [BOLD] Macro F1', '[BOLD] BBN  [BOLD] Micro F1']
On both benchmarks, the CLSC model achieves the best performance in all three metrics. When focusing on the comparison between NFETC and CLSC, we have following observation : Compact Latent Space Clustering shows its effectiveness on both clean data and noisy data. By applying CLSC regularization on the basic NFETC model, we observe consistent and significant performance boost; Hierarchical-aware loss shows significant advantage on the OntoNotes dataset, while showing insignificant performance boost on the BBN dataset. This is due to different distribution of labels on the test set. The proportion of terminal types of the test set is 69% for the BBN dataset, while is only 33% on the OntoNotes dataset. Thus, applying hierarchical-aware loss on the BBN dataset brings little improvement; Both algorithms are able to utilize noisy data to improve performance, so we would like to further study their performance in different noisy scenarios in following discussions.