END-TO-END NEURAL SEGMENTAL MODELS FOR SPEECH RECOGNITION
TABLE V: Average real-time factor per sample to compute gradients for different losses.
['[EMPTY]', 'pyramid', 'hinge', 'log loss', 'marginal log loss']
All numbers are measured on a single quad-core 3GHz CPU, averaged over the entire training set. As a reference, the average real-time factor for computing the gradients from the outputs of a regular 3-layer LSTM is 0.301, and reduces to 0.181 for pyramid 3-layer LSTM. Computing the gradient of the CTC loss is also 0.181. Though the exact number depends on the implementation, the general trend is clear. It is faster to compute the FC weight function than the SRNN weight function. Using the pyramid encoder significantly reduces the runtime. Computing the gradient of hinge loss is fastest, and computing the gradient of marginal log loss is the slowest. The real-time factor for decoding is 0.259 including the pyramid LSTM.