Guiding CTC Posterior Spike Timingsfor Improved Posterior Fusion and Knowledge Distillation
Table 2: WERs for knowledge distillation from BiLSTM to UniLSTM phone CTC models. [%]
['[EMPTY]', 'SWB', 'CH']
2A and 2B indicate the WERs by UniLSTM and BiLSTM phone CTC models trained with the standard procedure. As a naïve approach, in 2C, we trained a UniLSTM model through knowledge distillation from BiLSTM models with minimizing the frame-wise KL divergence, but saw degradation in WERs. In addition, posteriors from BiLSTM models trained with the standard training were not aligned and thus knowledge distillation from their posterior fusion was not successful.