N-ary Relation Extraction using Graph State LSTM
Table 3: Average test accuracies for Ternary drug-gene-mutation interactions. Single represents experiments only on instances within single sentences, while Cross represents experiments on all instances. *: significant at p<0.01
['Model', 'Single', 'Cross']
+multi-task applies joint training of both ternary (drug-gene-mutation) relations and their binary (drug-mutation) sub-relations. \newcitequirk-poon:2017: EACLlong use a statistical method with a logistic regression classifier and features derived from shortest paths between all entity pairs. Bidir DAG LSTM is our bidirectional DAG LSTM baseline, and GS GLSTM is our graph state LSTM model. Our final results are better than \newciteTACL1028, despite the fact that we do not use multi-task learning. Note that all systems show performance drops when evaluated only on single-sentence relations, which are actually more challenging. One reason may be that some single sentences cannot provide sufficient context for disambiguation, making it necessary to study cross-sentence context. Another reason may be overfitting caused by relatively fewer training instances in this setting, as only 30% instances are within a single sentence. One interesting observation is that our baseline shows the least performance drop of 1.7 points, in contrast to up to 4.1 for other neural systems. This can be a supporting evidence for overfitting, as our baseline has fewer parameters at least than FULL and EMBED.