Integrating whole context to sequence-to-sequence speech recognition
Table 1: Cloze Test Accuracy
['[EMPTY]', 'Accuracy']
5.4.1 Cloze Test Accuracy We first evaluate cloze test accuracy on AISHELL-1 test transcriptions. We compared four models. “UniCOR” means the COR model only has the forward stack, i.e. a standard transformer language model. “transcriptions” means that the model is trained on training transcriptions. The other two models are trained on the external text. From the table, we can see that the COR, which utilizes the whole context to predict words, achieves significantly higher accuracy than the UniCOR. This demonstrates that the prediction ability of the whole context. With the external text data, the larger scale one, COR achieves 0.996 of accuracy. Because the model trained on larger data achieves better performance, we use the model trained on external text in the following ASR experiments.