Improvements to Deep Convolutional Neural Networks for LVCSR
Table 3: WER as a function of # of hidden units
['Number of Hidden Units', 'WER']
Again the total number of parameters in the network is kept constant for all experiments. We can observe that as we increase the number of hidden units up to 220, the WER steadily decreases. We do not increase the number of hidden units past 220 as this would require us to reduce the number of hidden units in the fully connected layers to be less than 1,024 in order to keep the total number of network parameters constant. We have observed that reducing the number of hidden units from 1,024 results in an increase in WER. We were able to obtain a slight improvement by using 128 hidden units for the first convolutional layer, and 256 for the second layer. This is more hidden units in the convolutional layers than are typically used for vision tasks as many hidden units are needed to capture the locality differences between different frequency regions in speech.