ASAPP-ASR: Multistream CNN and Self-Attentive SRUfor SOTA Speech Recognition
Table 1: BPE-level perplexities on dev-clean and dev-other for 12 layer Transformer, 12 layer SRU and 24 layer SRU LMs. We include start and end of sentence tokens on each utterance.
['[BOLD] Model', '[BOLD] Layers', '[BOLD] # Params', '[BOLD] Dev clean', '[BOLD] Dev other']
With our 12 and 24-layer models, we achieve dev-clean perplexities of 36.2 and 34.3 respectively. In the first third of training we see the most improvement, with combined dev set perplexities at 40 for the 12-layer model and 37 for the 24-layer model.