Fine-tuning Pre-Trained Transformer Language Models to Distantly Supervised Relation Extraction
Table 1: Precision evaluated automatically for the top rated relation instances. † marks results reported in the original paper. ‡ marks our results using the OpenNRE implementation.
['System', 'AUC', 'P@100', 'P@200', 'P@300', 'P@500', 'P@1000', 'P@2000']
DISTRE with selective attention achieves a new state-of-the-art AUC value of 0.422. The results of the PCNN+ATT model indicate that its performance is only better in the very beginning of the curve, but its precision drops early and only achieves an AUC value of 0.341. Similar, RESIDE performs better in the beginning but drops in precision after a recall-level of approximately 0.25. This suggests that our method yields a more balanced overall performance, which we believe is important in many real-world applications. We again can observe that while DISTRE has lower precision for the top 500 predicted relation instances, it shows a state-of-the-art precision of 60.2% for the top 1000 and continues to perform higher for the remaining, much larger part of the predictions.