L-Vector: Neural Label Embedding for Domain Adaptation
Table 2: ASR WERs (%) of adapting a multi-conditional BLSTM acoustic model trained with 6400 hours US English to each of the 9 accented English and kids’ speech with one-hot label and the proposed NLE methods.
['Adapt. Method', 'A1', 'A2', 'A3', 'A4', 'A5', 'A6', 'A7', 'A8', 'A9', 'Kids']
This teacher model has 6 hidden layers with 600 units in each layer. 80-dimensional log Mel filterbank features are extracted from training, adaptation and test data. The output layer has 9404 units representing 9404 senone labels. The BLSTM is trained to minimize the frame-level cross-entropy criterion. There is no frame stacking or skipping. A 5-gram LM is used for decoding with around 148M n-grams. This well-trained source-domain model is used as the initialization for all the subsequent re-training and adaptation experiments. For accent adaptation, we train an accent-dependent BLSTM for each accented English using one-hot label with cross-entropy loss. Each accent-dependent model is trained with the speech of only one accent. For kids adaptation, we train a kids-dependent BLSTM using kids’ speech with one-hot labels. We use these results as the baseline. NLE-SKL performs the best among the three NLE adaptation methods, with 11.8%, 14.1%, 11.5%, 10.3%, 8.3%, 7.5%, 13.5%, 12.2%, and 5.4% relative WER reductions over the one-hot label baseline on A1 to A9, respectively. NLE-SKL consistently outperforms NLE-L2 and NLE-KL on all the accents, with up to 4.0% and 4.9% relative WER reductions over NLE-L2 and NLE-KL, respectively. The relative reductions for native and non-native accents are similar except for A9. NLE-KL performs slightly better than NLE-L2 on 6 out of 9 accents, but slightly worse than NLE-L2 on the other 3. All the three NLE methods achieve much smaller relative WER reductions (about 5%) on A9 than the other accents (about 10%). This is reasonable because North American English is much more similar to the source-domain US English than the other accents. The source-domain model is not adapted much to the accent of the target-domain speech. Parallel kids’ speech cannot be obtained through data simulation either. We adapt the 6400 hours BLSTM acoustic model to the collected real kids’ speech using NLE. NLE-SKL outperforms the other two NLE methods with a 6.0% relative WER reduction over the one-hot baseline. We find that NLE is more effective for accent adaptation than kids adaptation. One possible reason is that a portion of kids are at the age of teenagers whose speech is very similar to that of the adults’ in the 6400 hours source-domain data. Note that all the kids speech is collected in US and no accent adaptation is involved.