Efficient EUD Parsing
Table 8: Comparison of attachment scores for the development treebanks for distilled (dist) models and models with the same parameters (small) trained normally.
['[ITALIC]  [BOLD] Arabic', 'UAS', 'LAS', '[ITALIC]  [BOLD] Bulgarian', 'UAS', 'LAS']
For the most part the normal models outperform the distilled models. So perhaps without this extra information distillation is less effective. Also, dropout wasn’t used during distillation in the original paper but is here, so perhaps the values used here were too punitive a regularisation. Although we use the same hyperparameters as the original paper, the average LAS for the small normally trained models is 0.4 points less than the large model.