Denoising Distant Supervision for Relation Extraction via Instance-Level Adversarial Training
Table 2: Precision of various models for different recall (%).
['Method CNN+', 'Method ATT', '0.1 67.5', '0.2 52.8', '0.3 45.8', 'Mean 55.4']
We report the results of various neural architectures including CNN, PCNN, RNN and BiRNN with various denoising methods: +ATT is the selective attention method over instances Lin et al. ; +ADV is the denoising method by adding a small adversarial perturbation to instance embeddings Wu et al. We also compare our methods with feature-based models, including Mintz Mintz et al. The results of the baseline models all come from the data reported in their papers or their open-source code. Since we focus more on the performance of those top-ranked results, here we show the precision scores when the recall is 0.1, 0.2, 0.3 as well as their mean. We find that complicated neural models (PCNN, BiRNN) perform better than simple neural networks (CNN, RNN) when using the same denoising methods. Both CNNs and RNNs are significantly improved by adversarial training, and our method (AN) performs consistently much better than the adversarial training baseline (ADV). The improvements brought by changing denoising methods are more significant than the improvements brought by modifying neural models. This indicates that the wrong labeling problem is the critical factor that prevents distantly supervised RE models from working effectively.