Learning neural trans-dimensional random field language models with noise-contrastive estimation
Table 2: Speech recognition WERs on CHiME-4 Challenge data. “Dev” denotes the development set and “Test” denotes the test set. “real” denotes the speech recorded in real environments and “simu” denotes the simulated speech.
['model', 'Dev real', 'Dev simu', 'Test real', 'Test simu']
First, the LSTM LM achieves significant WER reduction compared with “KN5” with relative WER reduction 22.8% on real test set. Combining the LSTM LM and the n-gram LM leads to the state-of-the-art results, denoted by “KN5+LSTM2x512”. Building on top of the LSTM LM, our neural TRF LM can further reduce the WER with only 2-epoch NCE training. Remarkably, the lowest WER 5.44% on real test set is achieved by combining neural TRF LMs with n-gram LMs and LSTM LMs (denoted by “KN5+LSTM2x512+neural TRF”), with relative reduction of 4.7% compared to the strong state-of-the-art system “KN5+LSTM2x512”. This reveals that neural TRF LMs and various LMs in the directed graphical modeling approach are complementary, and a combination of them leads to WER reduction.