DeFINE: DEep Factorized INput Token Embeddings for Neural Sequence Modeling
(c) Comparison with existing works on the PTB dataset
['[BOLD] Model', '#  [BOLD] Parameters  [BOLD] (in millions)', '[BOLD] Perplexity  [BOLD] Val', '[BOLD] Perplexity  [BOLD] Test']
Though the adaptive input (baevski2018adaptive) and output ( grave2017efficientSoft) methods are effective and reduce the number of parameters significantly, our method further improves performance by about 3 points while learning only 1.25% (or 0.4 million) more parameters.