Sequence Prediction with Neural Segmental Models
Table 4.3: Phoneme error rates (%) with discriminative segmental cascades. The search space for the first pass is denoted H1, and the search space for the second pass H′1↑ is denoted H2, where H1 is pruned to get H′1.
['[EMPTY]', 'dev', 'test']
We use the sum of the lattice weight, the bigram LM weight, second-order boundary weights, segment NN weights, bi-phone NN bottleneck weights, length indicators, and lexicalized bias as our final weight function for the second level of the cascade. Hinge loss is minimized with AdaGrad for 20 epochs with step size 0.01. Again, no explicit regularizer is used except early stopping based on the PER on the development set. Adding the second-order weights, bigram LM, and the above NN weights gives a 1.8% absolute improvement over our best first-pass system, demonstrating the value of including such expensive features with wider context.