1 Introduction
Table 5: Document classification results, reporting number of parameters and accuracy.
['Representation', 'Parameters', 'Accuracy']
Our optimized representation is significantly smaller than the baselines, using 5x and 27x less parameters than the recurrent and convolutional baselines respectively. Even with far less capacity, the optimized model’s accuracy is comparable to the baselines. Specifically, it only drops by 1% in comparison to the recurrent baseline. In addition, each successively compressed convolutional model achieves an accuracy better than the baseline. This suggests that the original convolutional model is over-parameterized and the reduction in capacity helps during optimization. Our optimized model shows gains on the file size and latency metrics; most notably, its latency is an order of magnitude lower than the recurrent baseline’s. However, the CNNs all use slightly more memory than the recurrent model, presenting a clear trade-off between latency and memory requirements.