SelfORE: Self-supervised Relational Feature Learning for Open Relation Extraction
Table 1: Quantitative Performance Evaluation on three datasets.
['Dataset', 'Model', '[ITALIC] B3 F1', '[ITALIC] B3 Prec.', '[ITALIC] B3 Rec.', 'V-measure F1', 'V-measure Hom.', 'V-measure Comp.', 'ARI']
UIE-PCNN is considered as the previous state-of-the-art result. We enhance this baseline by replacing PCNN and GolVe embedding with the proposed BERT-based encoder and classifier. The enhanced state-of-the-art model, namely UIE-BERT, achieves the best performance among baselines. The proposed SelfORE model outperforms all baseline models consistently on B3 F1/Precision, V-measure F1/Homogeneity and ARI. SelfORE on average achieves 7.0% higher in B3 F1, 3.4% higher in V-measure F1 and 7.7% higher in ARI across three dataset when comparing with previous state-of-the-art. Unlike baseline methods which achieve high B3 Recall but low Precision, or high V-measure Completeness but low Homogeneity, our model obtains a more balanced performance while achieving the highest Precision and Homogeneity, although the B3 Recall and V-measure Completeness are less satisfactory. Having high precision and homogeneity scores can be a quite appealing property for precision-oriented applications in the real-world. More specifically, without self-supervised signals for relational feature learning, SelfORE w/o Classification gives us 14.4% less performance averaged over all metrics on all datasets. Similarly, Adaptive Clustering gives 6.2% performance boost in average over all metrics when comparing with the hard-assignment alternative (SelfORE w/o Adaptive Clustering).