Recurrent Memory Networks for Language Modeling
Table 2: Perplexity comparison including RMN variants with and without temporal matrix (tM) and linear (l) versus gating (g) composition function.
['[BOLD] Model', '[EMPTY]', '[BOLD] De', '[BOLD] It', '[BOLD] En']
All RMN variants largely outperform n-gram and MemN models, and most RMN variants also outperform the competitive LSTM baselines. The best results overall are obtained by RM with temporal matrix and gating composition (+tM-g).