Character-Aware Attention-Based End-to-End Speech Recognition
Table 1: The WER (%) performance of AED and CA-AED with different WSU output units for E2E ASR on a 3400 hours Microsoft Cortana dataset. Ne is the number of hidden layers in a encoder GRU and Np (in million) is the total number of model parameters. WERR (%) and PRR (%) are the relative WER improvement and the parameter reduction rate of a CA-AED with respect to the AED with the same Ne.
['WSU', 'System', '[ITALIC] Ne', 'WER', 'WERR', '[ITALIC] Np', 'PRR']
By predicting mixed-unit output, the WERs decrease to 9.31% and 7.58% with 4-layer and 6-layer encoders, respectively. AED achieves better ASR performance with mixed-unit output. We see that mixed-unit AED works slightly better than word-piece AED probably because the former one has more output units than the latter one. But the goal of this paper is not to compare the impact of the two output units on AED performance, but to show the benefits a WSU AED, regardless of the type of WSU, can get from the CA modeling. We vary the number of hidden layers in the encoder Ne to investigate the effectiveness of CA-AED for different model sizes with different parameter reduction rates (PRR). For mixed-unit model, CA-AED achieves 8.52% and 7.35% WERs, respectively, with 4-layer and 6-layer encoders, which are 8.5% and 3.0% relative improved over the AED baseline system with 29.3% and 26.0% reduction in model parameters, respectively.