Image Captioning with Sparse Recurrent Neural Network
Table 1: The effects of varying gating variable initialisation value m on MS-COCO. Sparsity level is set to 0.8. Bold text indicates best overall performance.
['Gating init. value', 'MS-COCO test set scores B-1', 'MS-COCO test set scores B-2', 'MS-COCO test set scores B-3', 'MS-COCO test set scores B-4', 'MS-COCO test set scores M', 'MS-COCO test set scores R', 'MS-COCO test set scores C', 'MS-COCO test set scores S']
From the table, we can see that the best overall performance is achieved when m is set to 5. Starting the gating parameters at a value of 5 allows all the captioning parameters θ to be retained with high probability at the early stages of training, allowing better convergence. This observation is also consistent with the works of [zhu2017prune] and [yu2019playing], where the authors found that gradual pruning and late resetting can lead to better model performance. Thus, we recommend setting m=5.0.