Phonetic and Graphemic Systems for Multi-Genre Broadcast Transcription
Table 5: %WER of phonetic and graphemic random ensembles of TDNN-LSTM models with and without model smoothing on dev17b.
['Model', 'Model', '%WER  [ITALIC] μ', '%WER  [ITALIC] σ', '%cWER', 'Ensemble Comb %WER', 'Ensemble Comb %Rel']
This is more expensive than model smoothing, but allows additional diversity to be introduced. LF-MMI training may benefit more from random ensemble generation, as it avoids the cross-entropy initialisation stage of approaches such as sMBR training, where the same targets are normally used for all system, possibly reducing the diversity of the final systems after sequence training. In this work, an ensemble of 3 TDNN-LSTM models was created by building 2 additional models using different seeds for random parameter initialisation. To put the cWER number in context, an ensemble of sMBR trained models on the AMI IHM task with a mean WER of 25% had a cWER of 12%. Given the large gains from model smoothing, it is interesting to examine ensembles of smoothed models. As expected the cWERs for the ensembles are reduced, as model smoothing reduces the diversity from the precise stopping points. However, there are still large gains of over 7% relative from the ensemble combination. Additionally, the difference between phonetic and graphemic, smoothed or unsmoothed, systems when combining random ensembles has been reduced to just 5% relative.