Sequence Prediction with Neural Segmental Models
Table 5.3: Frame error rates (%) of LSTM encoders after end-to-end fine-tuning with different losses .
['frame-wise cross entropy', 'frame-wise cross entropy', 'FER 17.8']
After end-to-end fine-tuning, the encoders are not constrained to perform well on frame-wise cross entropy. We evaluate the encoders on the frame classification task to see how much the encoders deviate from the initialization, and whether the intermediate representations trained for frame classification are still preserved. The encoders fine-tuned with hinge loss and log loss deviate less compared to the one fine-tuned with marginal log loss. We suspect this is because hinge loss and log loss use the ground-truth alignments in training while marginal log loss does not.