Heterogeneous Supervision for Relation Extraction: A Representation Learning Approach
Table 6: Number of relation mentions (RM), relation mentions annotated as None, relation mentions with conflicting annotations and conflicts involving None
['[BOLD] Dataset', '[BOLD] Wiki-KBP', '[BOLD] NYT']
It can be observed that a large portion of instances is only annotated as None, while lots of conflicts exist among other instances. This phenomenon justifies the motivation to employ true label discovery model to resolve the conflicts among supervision. Also, we can observe most conflicts involve None type, accordingly, our proposed method should have more advantages over traditional true label discovery methods on the relation extraction task comparing to the relation classification task that excludes None type. One could also observe the difference between REHession and the compared methods is more significant on the NYT dataset than on the Wiki-KBP dataset.