Language Modeling with Deep Transformers
Table 7: WERs (%) for hybrid systems on the LibriSpeech 960hr. 4-gram model is used in the first pass to generates lattices for rescoring. The row ”Lattice” shows oracle WERs of the lattices.
['LM', '[ITALIC] L', 'Para. in M', 'dev clean', 'dev clean', 'dev other', 'dev other', 'test clean', 'test clean', 'test other', 'test other']
We apply our word-level Transformer language models to conventional hybrid speech recognition by lattice rescoring. We obtain consistent improvements in terms of WER over the LSTM baselines.