Transformers with convolutional context for ASR
Table 2: WER for different decoder convolution architectures
['Conv. depth', 'Cxt size (num. kernels)', 'dev clean', 'dev other', 'test clean', 'test other']
To better understand these results, we also studied the effects of different hyperparameter settings. feature maps were chosen to ensure the total number of parameters are fixed between all configurations. The best performance comes from using the same parameter budget over wider context that is built over multiple convolutional layers. However, the decoder is able to get reasonable WER even with a context of just 3 words as input to the transformer layers.