RNN-T Models Fail to Generalize to Out-of-Domain Audio: Causes and Solutions
Table 3: WERs / deletions for Librispeech with regular inference (Reg.) and DOI using 16s window with 2s overlap.
['[EMPTY]', 'Reg.', 'DOI']
Librispeech: The prediction network has the same LSTM setup as the LAS decoder. The joint network has 640 hidden units, and uses the same word piece model. The results are shown in Tab. Despite achieving low WERs on the Librispeech test sets, the model exhibits high deletion errors on YT-short. DOI reduces the deletion error from 99.5% to 3.6%. The model still has a WER of 33.0% after using DOI, mainly due to substitutions (22.2%) caused by phonetically similar words. This is likely caused by the limited vocabulary the Librispeech model is exposed to during training.