End-to-end Training Approaches for Discriminative Segmental Models
Table 6: Phone error rates for segmental models trained end to end with dropout.
['[EMPTY]', 'dropout', 'dev', 'test']
Next, we train the same architecture end to end from scratch. We make sure that all the models are initialized identically to the two-stage systems. The four losses are used for training with dropout rates in {0,0.1,0.2,0.5}. Ground truth segmentations are used when training with hinge loss and log loss, and are disregarded when training with latent hinge loss and marginal log loss. The optimizers we use here are SGD with step sizes in {0.1,0.5}, momentum 0.9, and gradient clipping at norm 5, AdaGrad with step sizes in {0.01,0.02,0.04} and no clipping, and RMSProp with step size 0.001, decay 0.9, and no clipping. We run earch optimizer for 50 epochs with early stopping. First, all optimizers above fail to minimize latent hinge loss. All of them get stuck in local optima, and fail to produce reasonable forced alignments. Even though all loss functions in end-to-end training are nonconvex, latent hinge loss is more sensitive to initialization than other losses. The second observation is that adding dropout improves performance. However, using the same dropout rate as the two-stage system results in worse performance. Finally, though behind the best fine-tuned model, marginal log loss with dropout 0.2 slightly edges over other losses.