Highway Long Short-Term Memory RNNs for Distant Speech Recognition
Table 4: Performance of sequence training on various networks
['System (dr: dropout)', '#Layers', 'with overlap', 'no overlap']
The table suggests that introducing the highway connection between LSTMP layers is beneficial to sequence discriminative training. For example, without the highway connection, sequence training on top of the 3-layer LSTMP network brings WER from 50.7 down to 49.3, a relative improvement of only 3% on this particular task. After introducing the highway connection and dropout, the improvement is from 50.4 to 47.7, 5% relatively. The relative improvement is even larger on the non-overlapping segment subset, which is roughly 7%. The results suggest that sequence training is beneficial from both highway connection and a deeper structure.