RNN-T Models Fail to Generalize to Out-of-Domain Audio: Causes and Solutions
Table 2: WERs using streaming RNN-T models trained on Search data with variational noise on all layers (std=0.03) and SpecAugment uses 2 time masks and 2 frequency masks with widths up to 1.5 seconds and 27 dimensions, respectively.
['Models', 'Search', 'TTS-Audiobook', 'YT-short']
Streaming Models: Results are shown in Tab. As with the non-streaming models, the model with multiple regularizations gave the best improvements. SpecAugment+VN+RSP obtains a 76% improvement on TTS-Audiobook and a 62% improvement on YT-short. Other combinations also help, but are slightly worse than SpecAugment+VN+RSP. It should be noted that some of these models still perform better at 50k checkpoint. For example, SpecAug+VN obtains 14.5% and 22.5% on TTS-Audiobook and YT-short, respectively, at 50k steps. Although the combination doesn’t completely prevent overfitting, the degradation, as the model converges on the training data, is much lower than the baseline.