Tying Word Vectors and Word Classifiers:A Loss Framework for Language Modeling
Table 3: Comparison of our work to previous state of the art on word-level validation and test perplexities on the Penn Treebank corpus. Models using our framework significantly outperform other models.
['[BOLD] Model', '[BOLD] Parameters', '[BOLD] Validation', '[BOLD] Test']
The best LSTM model (VD-LSTM+REAL) outperforms all previous work which uses conventional framework, including large ensembles. when trained with reused embeddings (VD-RHN +RE) achieves the best overall performance, improving on VD-RHN by a perplexity of 2.5.