Learning Hidden Unit Contributions for Unsupervised Acoustic Model Adaptation
TABLE II: WER(%) for different re-parametrisation functions for LHUC transforms on TED tst2010. Unadapted baseline WER is 15.0%.
['[ITALIC] r', '2/(1+exp(− [ITALIC] r))', 'exp( [ITALIC] r)', 'max(0, [ITALIC] r)']
Finally, we explored how the form of the LHUC re-parametrisation function ξ affects the WER and frame error rate (FER) (Fig. For test-only adaptation only a small WER difference (0.1% absolute) is observed, regardless of the large difference in frame accuracies. This supports our previous observation that LHUC is robust against over-fitting. For SAT-LHUC training, a less constrained parametrisation was found to give better WERs for the SI model. Based on our control experiments, during SAT-LHUC training, setting ξ to be the identity function (linear r) gave similar results to ξ(r)=max(0,r) and ξ(r)=exp(r) and all were better than re-parametrising with ξ(r)=2/(1+exp(−r)). This is expected as for full training the last approach constrains the range of back-propagated gradients. From now on, if not stated otherwise, we will use ξ(r)=exp(r) in the remainder of this paper.