A Practical Framework for Relation Extraction with Noisy Labels Based on Doubly Transitional Loss
Table 1. Comparison of P@100, P@200, and P@300 and average precision (AP) for relation extraction for our model and the models in (Lin_ACL_2016) and (Wu_AAAI_2019)
['P@N (%)', '100', '200', '300', 'Mean', 'AP (%)']
For all of the baseline methods, we can see that the method in (Lin_ACL_2016) shows much better performance than the methods in (Mintz_ACL_2009), (Hoffmann_ACL_2011) and (Surdeanu_emnlp_2012), which demonstrates the effectiveness of the sentence-level selective attention. The method proposed in (Wu_AAAI_2019) has further improvement than the method in (Lin_ACL_2016) and other baseline methods. Although the method proposed in (Wu_AAAI_2019) has shown significant improvement over other baselines, our method still gains great improvement over the method in (Wu_AAAI_2019). Our method achieves the highest values for P@100, P@200, P@300, with mean value of 5.5 higher than the method in (Wu_AAAI_2019), and 14.6 higher than the method in (Lin_ACL_2016). Comparing to the metric P@100, P@200, P@300, our method has much more improvement on the metric of average precision. The reason can be intuitively summarized as following. Our method has much higher precision improvement when the recall is high than when the recall is low. We can see most the elements for the two matrices are similar, demonstrating the derived transitional matrix by the EM algorithm can successfully generalize to test data.