End-to-end Training Approaches for Discriminative Segmental Models
Table 5: Phone error rates for segmental models trained end to end initialized from two-stage systems trained with corresponding loss functions.
['[EMPTY]', 'dropout', 'dev', 'test']
Minimizing other losses from a model trained with hinge loss is less than ideal. We repeat the above experiments by warm-starting from a model trained with the loss function that we are going to minimize. We observe significant gains for log loss and marginal log loss if initialized with the matching loss function. Similarly, the gains with dropout in these cases are smaller than without dropout.