Compressing Word Embeddings via Deep Compositional Code Learning
Table 2: Reconstruction loss and the size of embedding layer (MB) of difference settings
['size (MB)', 'M=8', 'M=16', 'M=32', 'M=64']
Results: For different settings of the number of components M and the number of codewords K, we train the code learning network. For reference, we also report the total size (MB) of the embedding layer in the right table, which includes the sizes of the basis matrix and the hash table. We can see that increasing either M or K can effectively decrease the reconstruction loss. However, setting M to a large number will result in longer hash codes, thus significantly increase the size of the embedding layer. Hence, it is important to choose correct numbers for M and K to balance the performance and model size.