Letter-Based Speech Recognitionwith Gated ConvNets
Table 1: Architecture details. “#conv.” is the number of convolutional layers. Dropout amplitude, “#hu” (number of output hidden units) and “kw” (convolution kernel width) are provided for the first and last layer (all are linearly increased with layer depth). The size of the final layer is also provided.
['Dataset', 'Architecture', '#conv.', 'dropout first/last layer', '#hu first/last layer', 'kw first/last layer', '#hu full connect']
We tuned our acoustic model architectures by grid search, validating on the dev sets. We consider here two architectures, with low and high amount of dropout The amount of dropout, number of hidden units, as well as the convolution kernel width are increased linearly with the depth of the neural network. Convolutions are followed by a fully connected layer, before the final layer which outputs 30 scores (one for each letter in the dictionary). Concerning WSJ, the Low Dropout (p=0.2) architecture has about 17M trainable parameters. For LibriSpeech, architectures have about 130M and 208M of parameters for the Low Dropout (p=0.2) and High Dropout (p=0.2→0.6) architectures, respectively.