Image Captioning with Sparse Recurrent Neural Network
Table 6: Comparison of large-sparse and small-dense LSTM models.
['Models', 'NNZ parameters Sparsity', 'NNZ parameters Params.', 'NNZ parameters Overall CR', 'MS-COCO test set scores B-1', 'MS-COCO test set scores B-2', 'MS-COCO test set scores B-3', 'MS-COCO test set scores B-4', 'MS-COCO test set scores M', 'MS-COCO test set scores R', 'MS-COCO test set scores C', 'MS-COCO test set scores S']
In this section, we show that a large sparse LSTM image captioning model produced via end-to-end pruning is able to outperform a smaller dense LSTM trained normally. The small-dense model denoted as LSTM-S has a word embedding size of q=64 dimensions, LSTM size of r=128 units and finally attention MLP size of a=96 units. From the results, we can see that the small-dense model with 5× fewer parameters performs considerably worse than all the large-sparse models LSTM-M across the board.