TheanoLM — An Extensible Toolkit for Neural Network Language Modeling
Table 2: Development and evaluation set perplexities from the full-vocabulary models.
['Model', 'Perplexity Dev', 'Perplexity Eval']
Finnish vocabulary was five times larger, so the perplexities are considerably higher than in the English language experiment, as expected. The values are as reported by each tool; there might be differences in e.g. how unknown words and sentence starts and ends are handled. Perplexities of the Kneser-Ney models were computed using SRILM, which excludes unknown words from the computation. With TheanoLM we used the same behaviour (–unk-penalty 0).