Black-box Adaptation of ASR for Accented Speech
Table 4: Effect of changing service model
['Method', 'Indian G-US', 'Indian G-Video', 'Indian Jasper', 'Australian G-US', 'Australian G-Video', 'Australian Jasper']
In addition to the default Google Speech API service (G-US), we evaluate on two other models as service — a second Google speech-to-text model (G-Video) [GoogleASRVideo] meant for transcribing audio of video files, which works significantly better for MCV-v4 utterances because of their low-fidelity, and Jasper [li2019jasper], a recent end-to-end convolutional neural ASR model trained on the LibriSpeech dataset. We note here that we opted for G-US rather than Google’s ASR API for Indian English because of the latter’s poor performance (compared to G-US) on MCV-v4 utterances that are low bandwidth. WER of Local stays the same since service has no role during its training. We see a wide difference in accuracies across the different services. G-Video is the most accurate, but even in this case FineMerge is able to obtain a relative WER reduction by at least 3%. The Jasper model is worse than local Indian fine-tuned, yet FineMerge achieves more than 15% relative WER reduction wrt both service and local. This shows that the hyper-parameters of our service guided local inference adapt even to a weaker service.