Smaller Text Classifiers with Discriminative Cluster Embeddings
Table 5: IMDB test results for RNN with different embedding dimensions.
['Embedding Dimension', '1', '2', '3']
As the dimension increases, the hidden states become more spread out. The area ratio increases monotonically with increasing embedding dimension. The classification accuracy does not necessarily improve from larger usage of space. The reason for this could be the vanishing gradient problem in simple RNNs.