Semantic Relation Classification via Convolutional Neural Networks with Simple Negative Sampling
Table 2: Comparisons of our models with other methods on the SemEval 2010 task 8.
['Method', 'Feature Sets', 'F1']
For fair comparisons, we also add two types of lexical features, WordNet hypernyms and words around nominals, as part of input vector to the final softmax layer. Next, we will have a close look at the effect of our Negative Sampling method. We conduct additional experiments on the development set to compare two different negative sampling methods. For our proposed NS, we create a negative example from each non-other instance in the training set, 6,586 in total. We can see that our model still benefits from the randomly sampled negative examples, which may help our model learn to refine the margin between the positive and negative examples. However, with similar amount of negative examples, treating the reversed dependency paths from objects to subjects as negative examples can achieve a better performance (85.4% F1), improving random samples by 1.9%. This again proves that dependency paths provide useful clues to reveal the assignments of subjects and objects, and a model can learn from such reversed paths as negative examples to make correct assignments. Beyond the relation extraction task, we believed the proposed Negative Sampling method has the potential to benefit other NLP tasks, which we leave for future work.