Speaker Adaptation for End-to-End CTC Models
Table 5: Comparison of WERs (%) for various adaptation schemes using the word CTC mode on the 7-speaker set.
['Adapt. type', '# of utt.', 'Supervised KLD', 'Supervised MLT', 'Unsupervised KLD', 'Unsupervised MLT']
In general, both scalar and linear adaptations exhibit similar WER patterns to directly adapting the hidden layers in all of the operating conditions. For supervised adaptation, all three adaptation schemes produce significant improvements over the SI model by 7-8% on 200 development utterances (per speaker), even without regularization. The MTL adaptation further improves results by 2-3% relative WER. For unsupervised adaptation, the KLD regularization barely improves the results, even for low footprint adaptation models. This indicates that simply restricting the amount of model parameters could not alleviate the target sparsity issue. When the MTL criterion is employed, all three adaptation schemes yield 4-6% relative gain over the SI model on 200 development utterances. Both the linear adaptation and direct hidden layer adaptation benefit more from the MTL criterion than the scalar adaptation. In the following experiments, we will report results by directly updating the model parameters instead of inserting linear or scalar layers.