Deep Recurrent Neural Networks for Acoustic Modelling
Table 2: BLSTM WERs for Wall Street Journal. Larger recurrent models tend to perform better without overfitting. The deep BLSTM models do not yield any substantial gains over their single layer counterparts.
['[BOLD] Cell Size', '[BOLD] Layers', '[BOLD] dev93 WER', '[BOLD] eval92 WER']
Our deep BLSTM models contain two layers, the cell size reported is per direction per layer (e.g., total cells are quadrupled). Our deep BLSTM experiments give mixed results. For the same number of cells per layer, the deep model performs slightly better. However, if we fixed the number of parameters, the single layer BLSTM model performs slightly better, the single layer of 1024 bidirectional cells achieved a WER of 4.16 while the deep two layer BLSTM model with 512 bidirectional cells per layer achieved a WER 4.25.