Self-Attention Transducers for End-to-End Speech Recognition
Table 1: Comparisons of the SA-T with different weights.
['[BOLD] Model SA-T', '[ITALIC] β 0.0', '[BOLD] Dev 9.21', '[BOLD] Gain -', '[BOLD] Test 10.46', '[BOLD] Gain -']
We train SA-Ts using path-aware regularization with various weights β. We find that SA-T can achieve a CER of 10.46% on test set and the SA-Ts with regularization have consistently outperformed the plain SA-T. When β is 10, the model can achieve up to 11.1 % relative CER reduction compared to the plain SA-T. Learning curves in Fig. The results also indicate that SA-T can achieve better performance when the weight of regularization is great than 5. We suppose that a big wight can force the model to learn alignments better.