Sequence Prediction with Neural Segmental Models
Table 3.1: Letter error rates (%) for a baseline tandem HMM and segmental models trained with various losses and cost functions on the ASL data set.
['Tandem HMM', 'Tandem HMM', 'Tandem HMM', 'Andy 13.8', 'Drucie 7.10', 'Rita 26.1', 'Robin 11.5', 'Avg 14.6']
The evaluation metric is the letter error rate, which is the percentage of letters that are substituted, inserted, or deleted. We observe three consistent conclusions: Segmental models perform significantly better than the baseline HMM. Across losses, overlap cost is better than MPE cost. Hinge loss with overlap cost is the best performer, but this is only by a small margin, and log loss is very competitive even without using an explicit cost function. Non-convex losses (ramp and EBR) are difficult to optimize and therefore achieve inconsistent results. We suspect a warm start might be able to remedy this.