Jasper: An End-to-End Convolutional Neural Acoustic Model
Table 2: Normalization and Activation: Greedy WER, LibriSpeech after 50 epochs
['[BOLD] Model', '[BOLD] Normalization', '[BOLD] Activation', '[BOLD] Dev  [BOLD] Clean', '[BOLD] Dev  [BOLD] Other']
ReLU, clipped ReLU (cReLU), and leaky ReLU (lReLU) 2 types of gated units: gated linear units (GLU) We found that layer norm with GAU performed the best on the smaller model. Layer norm with ReLU and batch norm with ReLU came second and third in our tests. Using these 3, we conducted further experiments on a larger Jasper10x4. For larger models, we noticed that batch norm with ReLU outperformed other choices. Thus, leading us to decide on batch normalization and ReLU for our architecture.