Accelerating Natural Language Understandingin Task-Oriented Dialog
Table 2: ATIS performance of multi-task models compressed with structured pruning (ours) and knowledge distillation Hinton et al. (2015) as the compression rate (CR; %) increases. We report intent accuracy and slot F1. Darker shades of red indicate higher absolute performance drops with respect to 100%.
['Params', 'CR (%)', 'Pruning Intent', 'Pruning Slot', 'Distillation Intent', 'Distillation Slot']
Using a multi-task model on ATIS, we compress it with structured pruning and distillation, then examine its performance at varying levels of compression. Distillation achieves similar results as structured pruning with 0-50% sparsity, but its performance largely drops off after 80%. Surprisingly, even with extreme compression (99%), structured pruning is about 10% and 20% better on intents and slots, respectively.