Tensorized Embedding Layers
Table 4: CTR prediction. The hashed dataset is constructed as specified in Section 5.4 with hashing value 105. Embedding layers with more than 2000 unique tokens were replaced by TT–embeddings with shape factorizations consisting of 3 or 4 factors.
['[BOLD] Hash', '[BOLD] Model', '[BOLD] Factorization', '[BOLD] TT', '[BOLD] Hidden', '[BOLD] Test', '[BOLD] Emb.', '[BOLD] Total']
Besides that, we leave the overall structure of the neural network unchanged with the same parameters as in the baseline approach. To the best of our knowledge, our loss value is very close to the state-of-the-art result (juan2016field). These experiments indicate that the substitution of large embedding layers with TT–embeddings leads to significant compression ratios (up to 2011 times) with a slight improvement in the test loss, and up to 4200 with a small drop in the test loss. The total size of the compressed model does not exceed 20 Mb, while the baseline model weighs about 160 Mb. The obtained compression ratio suggests that the usage of TT–embedding layers may be beneficial in CTR prediction.