Connecting the Dots: Document-level Neural Relation Extraction with Edge-oriented Graphs
Table 1: Overall, intra- and inter-sentence pairs performance comparison with the state-of-the-art on the CDR test set. The methods below the double line take advantage of additional training data and/or incorporate external tools.
['Method', 'Overall (%) P', 'Overall (%) R', 'Overall (%) F1', 'Intra (%) P', 'Intra (%) R', 'Intra (%) F1', 'Inter (%) P', 'Inter (%) R', 'Inter (%) F1']
We directly compare our model with models that do not incorporate external knowledge. Verga et al. As it can be observed, the proposed model outperforms the state-of-the-art in CDR dataset by 1.3 percentage points of overall performance. We also show the methods that take advantage of syntactic dependency tools. Li et al. Our model performs significantly better on intra- and inter-sentential pairs, even compared to most of the models with external knowledge, except for Li et al. The first observation is that usage of direct edges only, reduces the overall performance almost by 4%, for inference step l=8. This drop mostly affects inter-sentence pairs, where a 18% point drop is observed. In fact, ordered edges (SSdirect) need longer inference to perform better, in comparison with additional indirect edges (SS) for which less steps are required. The superiority of SS edges, for all inference steps, compared to SSdirect edges on inter-sentence pairs detection, indicates that in a narrative, some intermediate information is not important.