Adversarial Teacher-Student Learning for Unsupervised Domain Adaptation
Table 2: The WER (%) performance of adversarial T/S learning adapted LSTM acoustic models for robust ASR on the real noisy channel 5 test set of CHiME-3. The adaptation data consists of “clean-noisy” and “clean-clean”.
['System', 'Conditions', 'BUS', 'CAF', 'PED', 'STR', 'Avg.']
To achieve environment-robust unsupervised adaptation, the condition classifier DNN Mc is designed to predict the posteriors of different environments at the output layer. As the adaptation data comes from both the clean and noisy environments, we first use an Mc with 2 output units to predict these two environments. The Nh and λ are fixed at 4 and 5.0 respectively in all our experiments. To achieve speaker-robust unsupervised adaptation, Mc is designed to predict the posteriors of different speaker identities at the output layer. The 7138 simulated and 1999 real noisy utterances in CHiME-3 training set are dictated by 83 and 4 different speakers respectively and the 9137 clean utterances are read by the same speakers. In speaker-robust adversarial T/S adaptation, an Mc with 87 output units are used to predict the posteriors of the 87 speakers. Larger WER improvement is achieved by speaker-robust unsupervised adaptation than the environment-robust methods. This is because T/S learning itself is able to reduce the environment variability through directly teaching the noisy student model with the senone posteriors from the clean data, which limits the space of improvement that environment-robust adversarial T/S learning can obtain. Speaker and environment robustness can be achieved simultaneously in unsupervised adaptation through MFA T/S learning, in which we need two condition classifiers: M1c predicts the posteriors of 87 speakers and M2c predicts the posteriors of 1 clean and 5 noisy environments in the adaptation data. The MFA T/S achieves lower WER than all the unifactorial adversarial T/S systems because it addresses the variations caused by all kinds of factors.