In Layman’s Terms: Semi-Open Relation Extraction from Scientific Texts
Table 3: Results on test set for our Rule-Based System (RBS) and SciIE Luan et al. (2018) w.r.t argument recognition and the combined task of extracting and classifying relations (RE). Providing the model with labels to distinguish trigger words from key-phrases slightly improves performance.
['[ITALIC] Argument Recognition', 'P', 'R', 'F1']
We evaluate SciIE on two sub-tasks: (1) Argument Recognition, and (2) Relation Extraction. With regards to the first sub-task, we train two SciIE models. One model only predicts whether a span is a valid span or not, while a second model predicts whether the span is a trigger word or a key-phrase. The RBS performs significantly worse; it identifies trigger words exceptionally well (F1=95.89 on test set) but does not correctly recognize many of the remaining key-phrases (F1=22.36 on test set), resulting in a low overall performance.