Log-Linear RNNs : Towards Recurrent Neural Networks with Flexible Prior Knowledge
Table 4: Log-perplexities per word (base e) of different models. The perplexity per word corresponding to a log-perplexity of 5.08 (resp. 6.07) is 161 (resp. 433).
['[EMPTY]', 'Training', 'Test1', 'Test2']
We use a notation such as LL-RNN (2500) to indicate a value M=2500 for the number of frequent word types considered as features. For each model, we stopped the training after the validation loss (not shown) did not improve for three epochs.