improving generalization of transformer for speech recognition with parallel schedule sampling and relative positional embedding
Table 3: Performance comparison in CER [%] when combining PSS and RPE.
['ExpID', 'SU', 'LU']
E9 is the combination of PSS and RPE with settings in second row of E3 and first row of E8, respectively. We assume the little improvement of PSS on LU is because we only perform the mixing process with K=1. B2 is TDNN-LSTM chain model trained with lattice free MMI on the same training set. Examining Fig. Compared to the base transformer, there is a 7.0% relative improvement of CER on SU and about 30% absolute gain for LU, which again confirms the effectiveness of our methods.