Deep Structured Neural Network for Event Temporal Relation Extraction
Table 7: Ablation over global constraints: Symmetry and Transitivity. Test is conducted on forward test set and both-way test set, which are denoted as −−−→Test and ←−→Test respectively. The local models trained on forward data and both-way data are denoted as →L and ↔L. Symmetry and Transitivity constraints are denoted as S and T. The results demonstrate that symmetry and transitivity constraints both improve model’s performance.
['[EMPTY]', 'TB-Dense −−−→ [ITALIC] Test', 'TB-Dense ←−→ [ITALIC] Test', 'Matres −−−→ [ITALIC] Test', 'Matres ←−→ [ITALIC] Test', 'TCR −−−→ [ITALIC] Test', 'TCR ←−→ [ITALIC] Test']
Impact of Implicit Symmetry (data augmentation): Examining the contrast between M1 and M3 as well as M2 and M4, we can see significant improvements in both-way evaluation despite slight performance drops in forward evaluation. These comparisons imply that data augmentation can help improve model robustness. Note that \newcitemeng2018context leveraged this data augmentation trick in their model. Impact of Explicit Symmetry: By comparing the proposed model with M4, the consistent improvements across all datasets demonstrate the benefit of using explicit symmetry property. Model Robustness: Although M1 and M2 show competitive results when evaluated on forward test data, their performance degrade significantly in the both-way evaluation. In contrast, the proposed model achieves strong performances in both test scenarios (best F1 scores except for one), and hence proves the robustness of our proposed method.