Mogrifier LSTM
Table 1: Word-level perplexities of near state-of-the-art models, our LSTM baseline and the Mogrifier on PTB and Wikitext-2. Models with Mixture of Softmaxes (Yang et al. 2017) are denoted with MoS, depth N with dN. MC stands for Monte-Carlo dropout evaluation. Previous state-of-the-art results in italics. Note the comfortable margin of 2.8–4.3 perplexity points the Mogrifier enjoys over the LSTM.
['[EMPTY]', '[EMPTY]', '[EMPTY]', 'No Dyneval Val.', 'No Dyneval Test', 'Dyneval Val.', 'Dyneval Test']
On the PTB and Wikitext-2 datasets, the Mogrifier has lower perplexity than the LSTM by 3–4 perplexity points regardless of whether or not dynamic evaluation (Krause et al. On both datasets, the state of the art is held by the AWD LSTM ( Merity et al. (Yang et al. The Mogrifier improves the state of the art without either of these methods on PTB, and without FRAGE on Wikitext-2.