Empirical Evaluation of Speaker Adaptation on DNN based Acoustic Model
Table 1: CERs of each speaker on baseline TDNN-LSTM i-vector based acoustic model. “S”, “M” and “H” are short forms for “slight”, “medium” and “heavy” separately.
['[BOLD] Speaker', '[BOLD] S01', '[BOLD] S02', '[BOLD] S03', '[BOLD] S04', '[BOLD] S05', '[BOLD] S06', '[BOLD] S07', '[BOLD] S08', '[BOLD] S09', '[BOLD] S10', '[BOLD] Avg']
We can see that the baseline model performs differently for each speaker and the average CER is 24.86%. The CER has a wide range from 3% to 56.62%. We manually checked the recordings from different speakers and found that speaker S05 had heavy accent and speaker S01 had slight Mandarin accent. This huge difference gives the speaker adaptation methods a big challenge. It’s necessary to compare different adaptation methods in terms of accent level. We believe that this is because the baseline model is trained using data mostly from Mandarin speakers with standard accent and the baseline model itself is robust enough; in this case, direct update on the network parameters may be harmful. RSI is still not stable and over-fitting happens when a large adaptation data set is used. If memory footprint is a major consideration, we suggest to use LHUC as its has a small set of adapted parameters for each speaker; otherwise LHUC and KLD can be both considered for medium-accent speakers. We believe that KLD’s superior performance is because the posterior distribution of the heavy-accent speech is far away from that of the unaccented speech; in this case, directly updating the network parameters or dragging the two distributions closer, is the most effective means. This also clarifies why RSI is better than LHUC and why we cannot observe over-fitting in this condition.