Jasper: An End-to-End Convolutional Neural Acoustic Model
Table 3: Sequence Masking: Greedy WER, LibriSpeech for Jasper 10x4 after 50 epochs
['[BOLD] Model', '[BOLD] Masking', '[BOLD] Dev  [BOLD] Clean', '[BOLD] Dev  [BOLD] Other']
During batching, all sequences are padded to match the longest sequence. These padded values caused issues when using layer norm. We applied a sequence mask to exclude padding values from the mean and variance calculation. Further, we computed mean and variance over both the time dimension and channels similar to the sequence-wise normalization proposed by Laurent et al. In addition to masking layer norm, we additionally applied masking prior to the convolution operation, and masking the mean and variance calculations in batch norm. Interestingly, we found that while masking before convolution gives a lower WER, using masks for both convolutions and batch norm results in worse performance.