Constructing Long Short-Term Memory based Deep Recurrent Neural Networks for Large Vocabulary Speech Recognition
Table 3: Speech recognition results of selected combinations for constructing deep LSTM networks.
['Model Descriptions', 'CER(%)']
For instance, a stacked LSTM-OPs network may be constructed by combining the deep hidden-to-hidden transition and the stack of LSTMs. Thus, experiments were conducted to evaluate some selected combinations of these methods for constructing deep LSTM networks, where each hidden layer had the same configuration as that in the experiments described above. However, the network, that had LSTM-OP layer on top of three feed-forward intermediate layers, yielded worse performance than the LSTM-OP network, which needed to be further researched. What is noteworthy is that the network that had three full-connection hidden layers on top of LSTM-OP layer yielded the best performance, and required less computations than the stacked LSTM-OPs network in both training and testing procedures.