Improving Relation Extraction with Knowledge-attention
Table 3: Macro-averaged F1 score on SemEval2010-Task8 dataset. ∗ marks statistically significant improvements over Self-attn with p<0.01 under one-tailed t-test.
['Model', 'mask entity', 'keep entity']
5.3.2 Results on SemEval2010-Task8 dataset We use SemEval2010-Task8 dataset to evaluate the generalization ability of our proposed model. Experiments are conducted in two manners: mask or keep the entities of interest. strategy degrades the performance, indicating that there exist strong correlations between entities of interest and relation classes in SemEval2010-Task8 dataset. Although the results of keeping the entities are better, the model tends to remember these entities instead of focusing on learning the linguistic clues of relations. This will result in bad generalization for sentences with unseen entities.