Learn Spelling from Teachers: Transferring Knowledge from Language Models to Sequence-to-Sequence Speech Recognition
Table 2: The perplexities on transcriptions of AISHELL-1 development set.
['LM', 'PPL']
5.3.1 The effectiveness of external text Firstly, we demonstrate the effectiveness of the external text data and the RNNLM. Note that the data is in character level, so the perplexities are relatively smaller. We can see that compared with the 3-gram with Kneser-Ney smoothing trained on training transcriptions, the 3-gram trained on external text achieves a significant reduction of perplexity. Moreover, the RNNLM achieves about a 28% relative reduction over the 3-gram trained on the external text.