FastText.zip:Compressing text classification models
Table 4: FlickrTag: Comparison of entropy pruning, norm pruning and max-cover pruning methods. We show the coverage of the test set for each method.
['Model #embeddings', 'full 11.5M', 'Entropy pruning 2M', 'Entropy pruning 1M', 'Norm pruning 2M', 'Norm pruning 1M', 'Max-Cover pruning 2M', 'Max-Cover pruning 1M']
We measure this effect on top of a fully quantized model. The full model misses 11.6% of the test set because of missing words (some documents are either only composed of hashtags or have only rare words). There are 312,116 labels and thus it seems reasonable to keep embeddings in the order of the million. A naive pruning with 1M features misses about 30−40% of the test set, leading to a significant drop of performance. On the other hand, even though the max-coverage pruning approach was set on the train set, it does not suffer from any coverage loss on the test set. This leads to a smaller drop of performance. If the pruning is too aggressive, however, the coverage decreases significantly.