Frame Stacking and Retaining for Recurrent Neural Network Acoustic Model
Table 3: 4-layers LSTM models with different number of stacked frames and matching frame retaining.
['[ITALIC] FS', '[ITALIC] FR', '[BOLD] CER', '[BOLD] RTF']
Frame stacking and retaining not only spends less time in training, but also brings faster decoder. Real time factor is utilized to evaluate the decoding speed. Neighboring frames have similar features, so there is no information omitted in stacking process. For our system, it is optimal that FS and FR are both set as 3. It reduces relative 41% RTF, and accuracy improves sightly.