Learn Spelling from Teachers: Transferring Knowledge from Language Models to Sequence-to-Sequence Speech Recognition
((b)) Varing λ for interpolation (T=5.0).
['Weight', 'CER%']
Then we fix T at 5.0 and evaluate the influence of λ. The parameter λ controls the proportion of the ground truth hard labels to the soft labels of the RNNLM. We can see that when λ=0.9, the model achieves the best performance on the development set. We refer to the model trained with λ=0.9 and T=5.0 as “Seq2Seq+LST” in the rest.