END-TO-END NEURAL SEGMENTAL MODELS FOR SPEECH RECOGNITION
TABLE I: Phone error rates (PER, %) on the development set for segmental models with different weight functions and different feature encoders. The encoders (pyramid or not) are trained with the frame-wise cross entropy. Two-stage training is denoted 2s, and two-stage training followed by fine-tuning is denoted 2s+ft. (* too slow to complete)
['FC', 'pyramid', '2s', 'hinge 19.9', 'log loss 20.7', 'marginal log loss 19.9']
For the FC weight function with a regular encoder, the three losses perform equally well, with marginal log loss having a slight edge over the other two. Using the pyramid encoder hurts the performance of hinge loss and log loss, but has less impact on marginal log loss. Hinge loss and log loss might be more sensitive to the reduced time resolution because they are tied to a specific segmentation, while marginal log loss is more forgiving due to the marginalization. Fine-tuning improves over two-stage training across all cases. The conclusion stays the same for the SRNN weight function, except that training the SRNN weight function without the pyramid is very time-consuming, and we only manage to complete a few epochs in the two-stage setting. Although the best results after fine tuning are roughly the same for both weight functions, to shorten the experimental cycle, we favor the better performer, the SRNN weight function, with a pyramid encoder in the two-stage setting.