Semantic Role Labeling for Learner Chinese: the Importance of Syntactic Parsing and L2-L1 Parallel Data
Table 3: Performances of the syntax-based and neural syntax-agnostic SRL systems on the L1 and L2 data. “ALL” denotes the overall performance.
['[EMPTY]', '[EMPTY]', 'PCFGLA-parser-based SRL Arg.-F', 'PCFGLA-parser-based SRL Adj.-F', 'PCFGLA-parser-based SRL F', 'PCFGLA-parser-based SRL ΔF', 'Neural-parser-based SRL Arg.-F', 'Neural-parser-based SRL Adj.-F', 'Neural-parser-based SRL F', 'Neural-parser-based SRL ΔF', 'Neural syntax-agnostic SRL Arg.-F', 'Neural syntax-agnostic SRL Adj.-F', 'Neural syntax-agnostic SRL F', 'Neural syntax-agnostic SRL ΔF']
For all systems, significant decreases on different mother languages can be consistently observed, highlighting the weakness of applying L1-sentence-trained systems to process learner texts. Comparing the two syntax-based systems with the neural syntax-agnostic system, we find that the overall ΔF, which denotes the F-score drop from L1 to L2, is smaller in the syntax-based framework than in the syntax-agnostic system. On English, Japanese and Russian L2 sentences, the syntax-based system has better performances though it sometimes works worse on the corresponding L1 sentences, indicating the syntax-based systems are more robust when handling learner texts.