Audio-attention discriminative language model for ASR rescoring
Table 3: Model size and Relative Word Error Rate Reduction (WERR) for different attention location
['[BOLD] Attention location', '[BOLD] # params (MM)', '[BOLD] WERR']
In a standard end-to-end systems, the attention to encoder states is applied at every layer. However, in a language model, the model has to learn both the dependency to acoustics as well as the temporal dependency in the word sequence. We see that adding audio context to the last LSTM-LM layer leads to some degradation in performance. However, using audio context at final affine layer or at the first LSTM layer leads to similar results.