Graph-to-Graph Transformer for Transition-based Dependency Parsing
Table 1: Results on English WSJ Treebank Stanford dependencies. * and ∘ indicate statistically significant improvement (α=0.01) over the model in the row above. Bold indicates not significantly different from the best result in that block.
['[EMPTY]', 'Test Set UAS', 'Test Set LAS', 'Dev Set UAS', 'Dev Set LAS']
Adding BERT initialisation results in large improvements for all the variants, with up to 32% LAS relative error reduction (RER), for G2GTr+H. This confirms that BERT initialisation helps Transformer with our graph input method, and supports the naturalness of inputting graph relations into the attention mechanism.