Applying GPGPU to Recurrent Neural Network Language Model based Fast Network Search in the Real-Time LVCSR
Table 2: Performance comparison of different types of LM rescorings
['Type Pass / Model', 'in-house WER', 'in-house xRT', 'eval92 WER', 'eval92 xRT', 'dev93 WER', 'dev93 xRT']
The one-pass types were computed by the proposed on-the-fly rescoring method, and the two-pass types were evaluated by rescoring the 1000-best hypotheses extracted from the lattices with 3-gram models. The two-pass hybrid type used interpolated score between 3-gram and RNNLMs. This paper explained how we applied RNNLMs to a real-time large vocabulary decoder by introducing the use of GPGPUs. We tried to accelerate RNNLM-based WFST traversals in GPGPU-CPU hybrid architectures by solving some practical issues for applying GPGPUs. Moreover, in order to minimize the computation burden on CPUs, we applied a cache strategy. The memory footprint for the cache method was small enough to perform the experiment on the large data set. However, it will be more desirable to employ efficient cache techniques to reduce the memory usage further.