Distant Supervision for Relation Extraction beyond the Sentence Boundary
Table 3: Average test accuracy in five-fold cross-validation. Uniform weights are used, except for adjacent-word edges.
['[BOLD] Paths', '[BOLD] Adj. Wt.', '[BOLD] Single-Sent.', '[BOLD] Cross-Sent.']
Next, we further evaluated the impact of path number and adjacency edge weight. Only dependency and adjacency edges were included in these experiments. Penalizing adjacency produces large gains; a harsh penalty is particularly helpful with fewer paths. These results support the hypothesis that dependency edges are usually more meaningful for relation extraction than word adjacency. Therefore, if adjacency edges get the same weights, they might cause some dependency sub-paths drop out of the top K paths, thus degrading performance. When the path number increases, there is a consistent and substantial increase in accuracy, which demonstrates the advantage of allowing adjacency edges to interleave with dependency ones. This presumably helps address syntactic parsing errors, among other things. The importance of adjacency weights decreases with more paths, but it is still significantly better to penalize adjacency edges.