Character-Level Language Modeling with Hierarchical Recurrent Neural Networks
Table 4: Perplexities of WLMs on the One Billion Word Benchmark in the literature
['Model', '# Params', 'PPL']
Also, our model performs better than LightRNN The model is basically a very large LSTM LM. However, the convolutional neural network (CNN) is used to generate word embedding of arbitrary character sequences as the input of the LSTM LM. Therefore, this model can handle OOV word inputs, however, still the model runs with a word-level clock.