Adaptive Input Representations for Neural Language Modeling
Table 7: Validation perplexity of our models on wikitext-103.
['[BOLD] Input', '[BOLD] Output', '[BOLD] Dropout', '[BOLD] Valid PPL', '[BOLD] Parameters']
This appendix shows various ablation. We also experimented with sharing the head projection but found this to perform less well than not sharing it.