Sequence Training of DNN Acoustic Models With Natural Gradient
Table 5: %WER differences between different optimisers on dev.sub2 with 158k trigram (200 hr)
['LM', 'CE', 'SGD', 'HF', 'DSAG-HF', 'NG']
Finally, these DNNs were tested on the dev.sub2 set as an evaluation set that was not used for setting training hyper-parameters to ensure that the trends observed above generalise. Furthermore these improvements have been fairly consistent between the validation dev.sub and evaluation dev.sub2 test sets. On average the NG method can be seen to provide approximately a 1% relative reduction in WER over both the SGD baseline and the DSAG-HF variant. While this improvement it fairly small, it is consistent and a statistical significance test (sign test of the word error rates at the episode level) showed that the improvement due to NG over each of the other methods is highly statistically significant (p<0.001).