Structurally Sparsified Backward Propagation for Faster Long Short-Term Memory Training
Table 3: Performance of trained image captioning models. Higher BLEU-4 score means better performance.
['Training Method', 'BLEU-4', 'Improvement']
Compared to the 5.92% performance loss in language modeling, this result demonstrates generality to applications beyond language model datasets. Moreover, the performance gap between the sparse training and the dense training can be mitigated by the dense after sparse approach. Interestingly, the 3-to-1 sparse-to-dense ratio used in the word language model also works well for the image captioning.