Improving the Performance of Online Neural Transducer Models
Table 6: WER for NT, Incorporating External LM
['System', 'Chunk', 'No LM', 'with LM']
Wpm + Lm Finally, we investigate incorporating an external LM into the MHA+WPM LAS and NT models. In these experiments, a n-gram FST LM, trained on 32K wordpieces, is used. This LM is trained on 1 billion text queries, a much larger set compared to the 15 million utterances seen by the LAS/NT models. Since the decoder of the LAS and NT models is an RNN-LM, it is possible there is nothing more to gain by incorporating the WPM FST. In the future, we will repeat this with a WPM RNN-LM trained on text data.