Exponential Moving Average Model in Parallel Speech Recognition Training
Table 2: The CER of LSTM and DNN final models trained with three methods.
['Methods', 'CER (%) LSTM', 'CER (%) DNN']
Since the EMA is a non-interference method, the performance can not be evaluated with the real-time FER. Therefore, the FER on validation sets are computed after every epoch. In order to present the decoding performance of MA-based methods, we extract 4 temporary models from each epoch to visualize the degradation of CER. It is significant that the frame accuracy of MA-based methods are higher than those of BMUF. Frame accuracies between MA-based methods have slight difference. It demonstrates that decoding result of EMA is always much better than that of BMUF, but that of MA fluctuates greatly, and even higher than that of BMUF sometimes. EMA method achieves about relative 3.9% CER reduction on test set, while MA method only achieves relative 2.1% CER reduction. Therefore, more accuracy models are trained with large-scale parallel training using EMA method, and it is more stable than MA method.