Document-Level N-ary Relation Extraction with Multiscale Representation Learning
Table 3: Comparison of our multiscale system with restricted variants and DPL Wang and Poon (2018) on CKB.777corrections in supplementary section B
['System', 'AUC', 'Max Recall', 'Precision', 'Recall', 'F1']
In all scenarios, our full model (MultiScale) substantially outperforms the prior state-of-the-art system (DPL). For example, in the best setting, using both noisy-or and the gene-mutation filter, the full model improves over DPL by 8.4 AUC points. Multiscale learning is the key to this performance gain, with MultiScale substantially outperforming more restricted variants. Not surprisingly, expanding extraction scope from sentences to paragraphs resulted in the biggest gain, already surpassing DPL. Conducting end-to-end learning over a document-level representation, as in DocLevel, is beneficial compared to ensembling over predictions for individual discourse units (SentLevel, ParaLevel), especially in the base version. Interestingly, MultiScale still attained significant gain over DocLevel with an ensemble over SentLevel and ParaLevel, suggesting that the document-level representation can still be improved. In addition to prediction accuracy, the document-level models also have much more room to grow, as maximum recall is about 20 absolute points higher in MultiScale and DocLevel, compared to ParaLevel or DPL. P=precision, R=recall. Compared to standard sentence-level extraction, our method can extract relations among entities that never co-occur in the same sentence or even paragraph. MultiScale can substantially improve the recall by making additional cross-sentence and cross-paragraph extractions. Often, a paper would first describe the mutations present in a patient cohort, and later describe the effects of drug treatment. There are also instances of bridging anaphora, for example via cell lines. One paper first stated the gene and mutation for a cell line “The FLT3-inhibitor resistant cells Ba/F3-ITD+691, Ba/F3-ITD+842, …, which harbored FLT-ITD plus F691L, Y842C, …mutations…”, and later stated the drug effect on the cell line “ E6201 also demonstrated strong anti-proliferative effects in FLT3-inhibitor resistant cells…such as Ba/F3-ITD+691, Ba/F3-ITD+842 …”.