A Practical Framework for Relation Extraction with Noisy Labels Based on Doubly Transitional Loss
Table 4. Comparison of P@1000 and average precision (AP) for three methods: BERT, Relation_BERT, and Relation_BERT+Transition_Loss
['Methods', 'P@1000 (%)', 'AP']
From this figure, we can see that the general method BERT has the much worse performance than the other two methods. Although BERT has shown very strong performance in general text classification (bert_Jacobv_corr_bert_2018), but without incorporating the locations of the target entities and their corresponding output hidden vectors, it does not show any advantage on this task. On the other hand, Relation_BERT shows much stronger performance than general BERT, which demonstrates the effective way that we incorporate the target entity information into the model. Furthermore, the approach Relation_BERT+Transition_Loss which incorporate all components in our model architecture gains significant improvement over Relation_BERT. We can see that when the recall is 0.3, Relation_BERT+Transition_Loss has the precision value of 0.74, while Relation_BERT has the precision value of 0.6, which shows 14% precision improvement on this recall point. Relation_BERT+Transition_Loss has the highest P@1000 value, with 11.6% absolute improvement over Relation_BERT, and 38.8% absolute improvement over BERT.