A Syntax-aware Multi-task Learning Framework for Chinese Semantic Role Labeling
Table 1: Experimental results of syntax-aware methods we compare on CPB1.0 dataset.
['Methods', 'Dev P', 'Dev R', 'Dev F1', 'Test P', 'Test R', 'Test F1']
Results of Syntax-aware Methods. First, the first line shows the results of our baseline model, which only employs the word embeddings and char representations as the inputs of the basic SRL model. Second, the Tree-GRU method only achieves 80.06 F1 score on the test data, which even didn’t catch up with the baseline model. We think this is caused by the relatively low accuracy in Chinese dependency parsing. Third, the FIR approach outperforms the baseline by 2.17 F1 score on the test data, demonstrating the effectiveness of introducing fixed implicit syntactic representations. Forth, the HPS strategy achieves more significant performance by 83.51 F1 score. Finally, our proposed framework achieves the best performance of 83.91 F1 score among these methods, outperforming the baseline by 3.43 F1 score. All the improvements are statistically significant (p<0.0001). From these experimental results, we can conclude that: 1) the quality of syntax has a crucial impact on the methods which depend on the systematic dependency trees, like Tree-GRU, 2) the implicit syntactic features have the potential to improve the down-stream NLP tasks, and 3) learning the syntactic features with the main task performs better than extract them from a fixed dependency parser.