Document-Level N-ary Relation Extraction with Multiscale Representation Learning
Table 6: Results of subrelation decomposition baselines on CKB, with the gene-mutation filter. MR=max recall, P=precision, R=recall.
['System', 'AUC', 'MR', 'P', 'R', 'F1']
As a baseline, we also consider a different document-level strategy where we decompose the n-ary relation into subrelations of lower arity, train independent classifiers for them, then join the subrelation predictions into one for the n-ary relation. We found that with distant supervision, the gene-mutation subrelation classifier was too noisy. Therefore, we focused on training drug-gene and drug-mutation classifiers, and joined each with the rule-based gene-mutation predictions to make ternary predictions. The paragraph-level drug-mutation model is quite competitive, which benefits from the fact that the gene-mutation associations in a document are unique. This is not true in general n-ary relations. Still, it trails MultiScale by a large margin in predictive accuracy, and with an even larger gap in the potential upside (i.e., maximum recall). The drug-gene model has higher maximum recall, but much worse precision. This low precision is expected, as it is usually not valid to assume that if a drug and gene interact, then all possible mutations in the gene will have an effect on the drug response.