AIPNet: Generative Adversarial Pre-training of Accent-Invariant Networks for End-to-End Speech Recognition
Table 1: WER (%) of different approaches in each accent in supervised setting. F1 indicates fine-tuning with LASR; F2 indicates fine-tuning with L′G; AI indicates accent-independent model; AD indicates accent-dependent model.
['Approach Baselines', 'Approach B1', 'Approach AI', 'Ave. 8.7', 'US 5.7', 'Non-US 9.0', 'CA 6.4', 'FR 8.4', 'IN 11.2', 'KR  [BOLD] 9.9', 'PH 7.2', 'LA  [BOLD] 7.8', 'GB 8.0', 'VN 13.0']
In our approach, we report results of fine-tuning GAI and LAS with LASR using transcriptions (F1), as well as those of fine-tuning the entire network with L′G using both transcriptions and accent labels (F2). We can see that fine-tuning with LASR (F1) outperforms the baselines by 2.3∼4.5% relative reduction on average WER. Compared to all the baselines, F1 has achieved improvement in CA, FR, GB, and especially IN (9.1∼15.3% reduction) but has shown a mediocre performance in each of the remaining accents. Reducing overfitting We further investigate the trend of word piece validation accuracy of the ASR model in B1 and F2, as shown in Fig. Compared to B1, F2 learns more slowly and reaches a better local optimal. This observation corroborates the effectiveness of the regularization in our approach on reducing the risk of overfitting. The sufficient training data with high-quality transcriptions in supervised setting empowers the learning capability of the ASR model. As a result, the additional regularization might even weaken such learning capability.