Graph-to-Graph Transformer for Transition-based Dependency Parsing
Table 3: Labelled attachment score on 13 UD corpora for [1] kulmizev2019deep with BERT pre-training and two of our models.
['Lang', '[1]', 'BERT DepTr+CH', 'BERT G2GTr+H']
As the baseline, we use scores of the model proposed by \newcitekulmizev2019deep, which uses the deep contextualised word representations of BERT and ELMo Peters_2018 as additional input to their parsing models. Our BERT G2GTr+H model performs better on 11 languages, which confirms again the effectiveness of our Graph2Graph Transformer architecture. Since the proposed model of \newcitekulmizev2019deep also contains a character-level BiLSTM architecture, which builds additional character-level input embeddings, this model performs better on Turkish and Korean, which are morphologically rich languages. We leave to future work the investigation of different mechanisms to input tokens and handle lexical information.