Neural Semi-Markov Conditional Random Fields for Robust Character-Based Part-of-Speech Tagging
Table 2: Tokenization and joint token-POS F1 on UD v2.0. Best scores are in bold, second-best are underlined.
['[EMPTY]', 'UDPipe 1.2 Tokens', 'UDPipe 1.2 POS', 'Stanford Tokens', 'Stanford POS', 'FBAML Tokens', 'FBAML POS', 'TRL Tokens', 'TRL POS', 'IMS Tokens', 'IMS POS', 'Our Tokens', 'Our POS']
Multilingual Results (UD v2.0). While for each language another shared task system performs best, our system performs consistently well across languages (best or second-best except for EN), leading to the best average scores for both tokenization and POS tagging. Moreover, it matches the state of the art for Chinese (ZH) and Vietnamese (VI), two languages with very different characteristics in tokenization. Evaluation. With the latter, we can assess the performance of MarMot without penalizing it for potential errors of UDpipe. For calculating the relaxed accuracy, we count the POS tag of a gold token as correct if MarMot predicts the tag for any subpart of it. We provide more details on the relaxed evaluation (description, examples and implementation) in our code repository. Note that we apply the relaxed evaluation only to UDpipe+MarMot but not to our model. The output of our model is directly evaluated against the gold labels of the clean corpus.