AIPNet: Generative Adversarial Pre-training of Accent-Invariant Networks for End-to-End Speech Recognition
Table 2: WER (%) of different approaches in each accent in semi-supervised setting. F1 indicates fine-tuning with LASR; F2 indicates fine-tuning with L′G; PL indicates pseudo labeling; AI indicates accent-independent model; AD indicates accent-dependent model.
['Approach w/o PL', 'Approach Baseline', 'Approach B1', 'Approach AI', 'Ave. 29.9', 'US 10.6', 'Non-US 31.8', 'CA 22.0', 'FR 33.1', 'IN 41.0', 'KR 33.1', 'PH 28.2', 'LA 28.7', 'GB 28.3', 'VN 40.6']
, we compare the performance between B1 and F1. As B2, B3, B4 and F2 require the availability of pairs of transcriptions and accent labels for training, the results of these approaches are not available in such scenario. The results have shown that our approach significantly outperforms the baseline model in all accents, achieving 3.4∼11.3% relative WER reduction. To achieve better performance, we generate pseudo transcriptions for non-US training data using the US model. By comparing the performance between models with and without pseudo labeling, we can observe that pseudo labeling has shown significant gains for all the approaches and almost in each accent, exhibiting its effectiveness on improving generalization performance using unlabeled data. In the scenario with pseudo labeling, fine-tuning with L′G (F2) outperforms the baselines by 1.6∼6.1% relative reduction on average WER and consistently achieves the best performance in all non-US accents except for CA.