END-TO-END NEURAL SEGMENTAL MODELS FOR SPEECH RECOGNITION
TABLE III: Segmental models trained end to end with different losses compared with the CTC model.
['[EMPTY]', 'dev', 'test']
In this section, we compare losses for end-to-end training of segmental models with the pyramid encoder and the SRNN weight function. Unlike in multi-stage training, all of the experiments here are trained from random initialization. While the log loss and marginal log loss achieve reasonable performance, the hinge loss completely fails. We find that hinge loss values on the training set are higher compared to the multi-stage models, suggesting that there is an optimization issue. Since log loss can be minimized reasonably well, we suspect that hinge loss is difficult to minimize because of its non-smoothness. The result with the marginal log loss is consistent with reported numbers in previous work The performance of CTC is on par with the segmental model trained with marginal log loss.