Self-Attention Transducers for End-to-End Speech Recognition
Table 2: Comparisons of the SA-T with different length contexts.
['[BOLD] Model', '[ITALIC] Nl', '[ITALIC] Nr', '[BOLD] Dev', '[BOLD] Test']
It is easy to observe that the longer contexts be modeled, and the better performances model have, which is consistent with our common sense. Chunk-flow SA-T with 20 left-context frames and 10 right-context frames has the best performance that is competitive to baseline RNN-T. The results indicate that chunk-flow mechanism can model long-term dependencies by stacking multiple blocks and implement online decoding without too much degradation of performance. Furthermore, we also train a chunk-flow SA-T with path-aware regularization (β=10), and observe that the model get 12.5% and 11.4% relative CER reduction on dev and test set respectively compared to the one without regularization. It proves the effectiveness of path-aware regularization again.