Czech Text Processing with Contextual Embeddings: POS Tagging, Lemmatization, Parsing and NER
Table 5: Czech PDT UD 2.3 results for POS tagging (UPOS: universal POS, XPOS: language-specific POS, UFeats: universal morphological features), lemmatization and dependency parsing (UAS, LAS, MLAS, and BLEX scores). Bold indicates the best result, italics related work.
['WE', 'BERT', 'Flair', 'UPOS', 'XPOS', 'UFeats', 'Lemmas', 'UAS', 'LAS', 'MLAS', 'BLEX']
This treebank is derived from PDT 3.5 a-layer, with original POS tags kept in XPOS, and the dependency trees and lemmas modified according to UD guidelines. In the previous ablation experiments, we used the gold segmentation and tokenization in the Czech PDT UD 2.3 treebank. For comparison with state of the art, Czech PDT UD 2.2 treebank without gold segmentation and tokenization is used in evaluation, according to the CoNLL 2018 shared task training and evaluation protocol. Our system reuses segmentation and tokenization produced by UDPipe 2.0 in the CoNLL 2018 Our sequence-to-sequence (seq2seq) model which captures the nested entities, clearly surpasses the current Czech NER state of the art. Furthermore, significant improvement is gained when adding the contextualized word embeddings (BERT and Flair) as optional input to the LSTM encoder. The strongest model is a combination of the sequence-to-sequence architecture with both BERT and Flair contextual word embeddings.