Essence Knowledge Distillation for Speech Recognition
Table 2: Word error rates of different models trained with a subset of the Switchboard data.
['Acoustic Model', '[ITALIC] k', 'SWB', 'CHE', 'TOTAL']
A subset consisting 25% of the training data from the Switchboard data set was used to quickly evaluate the effectiveness of the proposed method and to tune some hyperparameters. As can be seen, the TDNN-LSTM performed better than the TDNN model. The teacher model, which is a fusion of a TDNN model and a TDNN-LSTM model, significantly outperformed any individual model.