Structured Sparsification of Gated Recurrent Neural Networks
Table 2: Quantitative results for the unstructured Bayesian sparsification. Bayes W corresponds to the SparseVD method of emnlp. For language modeling, we evaluate quality on validation and test sets. Compression is equal to |W|/|W≠0|. In the last columns, the numbers of the remaining hidden neurons and non-constant gates in the LSTM layers are reported.
['Task', 'Method', 'Quality', 'Compression', 'Neurons', 'Gates']
In this section, we present experimental results for the unstructured Bayesian sparsification (configuration Bayes W). This configuration corresponds to a model of emnlp. Since Bayes W model does not comprise any group weights, the overall compression of the RNN is lower than for Bayes W+G+N (tab. However, the patterns in gate structures are the same as in Bayes W+G+N gate structures (fig.