First-Pass Large Vocabulary Continuous Speech Recognition using Bi-Directional Recurrent DNNs
Table 2: Train and test set character error rate (CER) results for a deep neural network (DNN) without recurrence, recurrent deep neural network with forward temporal connections (RDNN), and a bi-directional recurrent deep neural network (BRDNN). All models have 5 hidden layers. The DNN and RDNN both have 2,048 hidden units in each hidden layer while the BRDNN has 1,824 hidden units per hidden layer to keep its total number of free parameters similar to the other models. For all models we choose the most likely character at each timestep and apply CTC collapsing to obtain a character-level transcript hypothesis.
['Model', 'Parameters (M)', 'Train CER', 'Test CER']
Previous experiments with DNN-HMM systems found minimal benefits from recurrent connections in DNN acoustic models. It is natural to wonder whether recurrence, and especially bi-directional recurrence, is an essential aspect of our architecture. To evaluate the impact of recurrent connections we compare the train and test CERs of DNN, RDNN, and BRDNN models while roughly controlling for the total number of free parameters in the model.