Towards a Competitive End-to-End Speech Recognition for CHiME-6 Dinner Party Transcription
Table 4: External NNLM for RNN-T beam search
['[BOLD] NNLM', '[BOLD] WER(%)']
We also faced a problem in the rescoring process using external RNN-based language models. Character and word-based LSTM LMs (1-layer 1024-units) were trained on the training data transcriptions. However, the use of these LMs for hypothesis rescoring in the beam search algorithm for RNN-T only led to a decrease in recognition accuracy. We also used a pretrained AWD-LSTM-LM, which has shown significant WER improvement during lattice rescoring in the STC CHiME-6 system [Medennikov_chime6]. Unfortunately, none of these models improved accuracy. We have two ideas about this. The first one is that in case of a low-resource task, a regular RNN-T with well-chosen parameters of the Predictor network is able to show sufficiently good results without any rescoring techniques. The second one is that the beam search algorithm we used is not optimal for the hypothesis rescoring. For example, in the case of a hybrid system, there are separate acoustic and language model scores of a word unit. And in the rescoring algorithm, the language score is re-weighted according to the external LSTM LM. But in the case of RNN-T, there is only one single score for word unit. The default ESPnet rescoring just adds the weighted score of external LSTM LM to RNN-T weight. We used the beam size of 10 and NNLM weight of 0.3 for all experiments.