Smaller Text Classifiers with Discriminative Cluster Embeddings
Table 3: IMDB test results. The four rows above the dashed line are from Shu and Nakayama (2018); our results are below it.
['[EMPTY]', 'embedding', 'model', 'acc.']
Compared with compositional coding, our models perform much better with a much smaller set of embedding parameters even when we use a smaller number of cluster embeddings (e.g., compare 8 × 8 coding to CE; both use a comparable number of cluster embedding vectors, while CE works better). ME (with k=50 clusters and unique embeddings for the u=300 most frequent words) outperforms all other models while retaining a small model size. CAE performs better than SE, but uses more parameters than CE. We find a better trade-off with ME, which only adds parameters to the most frequent words.