Distilling an Ensemble of Greedy Dependency Parsers into One MST Parser
Table 5: Dependency parsing performance on English, Chinese, and German tasks. The “P?” column indicates the use of pretrained word embeddings. Reranking/blend indicates that the reranker score is interpolated with the base model’s score. Note that previous works might use different predicted tags for English. We report accuracy without punctuation for English and Chinese, and with punctuation for German, using the standard evaluation script in each case. We only consider systems that do not use additional training data. The best overall results are indicated with bold (this was achieved by the ensemble of greedy stack LSTMs in Chinese and German), while the best non-ensemble model is denoted with an underline. The † sign indicates the use of predicted tags for Chinese in the original publication, although we report accuracy using gold Chinese tags based on private correspondence with the authors.
['[BOLD] System', '[BOLD] Method', '[BOLD] P?', '[BOLD] PTB-SD  [BOLD] UAS', '[BOLD] PTB-SD  [BOLD] LAS', '[BOLD] CTB  [BOLD] UAS', '[BOLD] CTB  [BOLD] LAS', '[BOLD] German CoNLL’09  [BOLD] UAS', '[BOLD] German CoNLL’09  [BOLD] LAS']
Accuracy. First, consider the neural FOG parser trained with Hamming cost (CH in the second-to-last row). This is a very strong benchmark, outperforming many higher-order graph-based and neural network models on all three datasets. Nonetheless, training the same model with distillation cost gives consistent improvements for all languages. For English, we see that this model comes close to the slower ensemble it was trained to simulate. For Chinese, it achieves the best published scores, for German the best published UAS scores, and just after \newcitebohnet_nivre_12 for LAS.