Graph-to-Graph Transformer for Transition-based Dependency Parsing
Table 2: Comparisons to SoTA on the test set of English WSJ Treebank Stanford dependencies.
['Models', 'UAS', 'LAS']
Both with and without BERT initialisation, the same pattern appears across variants. Adding an embedding of the history sequence of parser actions (DepTr+H) results in a significant improvement (34%/33% LAS RER without/with BERT) compared to the DepTr model. Adding explicit modelling of the composition of sub-phrases (DepTr+CH) results in further improvement with BERT (8.73% LAS RER). The most directly comparable model is the stack-LSTM model of \newcitedyer-etal-2015-transition, which we used as the basis for our model. The G2GTr model reaches better accuracy and adding BERT greatly improves over the stack-LSTM. Other models include techniques that we could easily add to our model, such as beam search andor-etal-2016-globally; weiss-etal-2015-structured, dynamic oracle ballesteros-carlini-2013-maltdiver; kiperwasser-goldberg-2016-simple, and joint training with a PoS tagger yang2017joint. Nonetheless, compared to the previous state-of-the-art, our BERT G2GTr and BERT G2GTr+H models perform significantly better.