Improved training of neural trans-dimensional random field language models with dynamic noise-contrastive estimation
Table 1: Speech recognition results of various LMs, trained on WSJ portion of PTB dataset. “PPL” is the perplexity on PTB test set. “WER” is the rescoring word error rate on WSJ’92 test data. “#param” is the number of parameter numbers (in millions). “Training time” denotes the total training time for a LM. “Inference time” denotes the average time of rescoring the n-best list for each utterance.
['Model', 'PPL', 'WER (%)', '#param (M)', 'Training time', 'Inference time']
Speech recognition results on HKUST Chinese dataset. “Valid” is the character error rate (CER) on the valid set and “Test” is the CER on the test set. “+” denotes the log-linear interpolation with equal weights. As the vocabulary size is not large, all LSTM LMs use the standard softmax output layers to ensure the best model performance. The conclusions of this experiments are summarized as follows.