Exploring Pre-training with Alignments for RNN Transducer based End-to-End Speech Recognition
Table 3: WER comparison of different encoder pre-training methods on the task with 65,000 hours production training data.
['Model', 'WER (%)']
From our experiments, both the encoder pre-training and the whole-network pre-training can improve the performance of RNN-T model. In order to get more convincing results, we evaluate our proposed methods on very large scale data, where we use the 65,000 hours Microsoft production data set. Due to the very large resource requirement and computation cost, we only evaluate CE-based encoder pre-training method, which obtained the best accuracy in Cortana experiments. All the results are obtained using beam search, and the beam width is 5.