Improving speech recognition by revising gated recurrent units
Table 2: Comparison of the per-epoch training time for the RNN architectures optimized on the TIMIT development set.
['Architecture', 'Layers', 'Neurons', '# Params', 'Training time']
Results confirm that the main advantage of the proposed model is its ability to significantly reduce the training time. In our Theano implementation, running each GRU epoch lasts 580 seconds on an NVIDIA K40 GPU against just 390 seconds taken by the M-reluGRU, with a training time reduction of more than 32%. Results show that for GRU models the best performance is achieved with 5 hidden layers of 465 neurons.