Character-Level Language Modeling with Hierarchical Recurrent Neural Networks
Table 3: Perplexities of the HRNN CLMs on the One Billion Word Benchmark
['Model', 'Size', '# Params', 'BPC', 'Word PPL']
Due to the large amount of training data and weeks of training time, only two HLSTM-B experiments are conducted with the size of 4x512 and 4x1024. Therefore, further improvement in perplexity can be expected with bigger networks.