Length-Controllable Image Captioning
Table 3: Performance analysis on 4-level LaBERT. “Speedup” indicates the relative speedup over an autoregressive baseline.
['Method Lvl 4', 'Method 10 refine steps', 'S 21.6', 'C 39.5', 'M 27.3', 'R 46.9', 'B@1 55.9', 'B@4 19.0', 'Speedup ×2.5']
In this section, we analyze the effect of some hyper-parameters in 4-level LaBERT, i.e., the number of refine steps T, the [EOS] decay factor γ in Eqn. To analyze the effect of refine steps T, we vary T from 10 to 25 for caption generation on the fourth level; to show the importance of the [EOS] decay factor γ and the global update rule, we perform ablation experiments for them on the second level. From the table, the autoregressive baseline, i.e., 4-level VLP performs slightly better on SPICE; however, it requires 25 steps to decode the entire sequence. On the contrary, our non-autoregressive LaBERT can use a much smaller T to achieve acceleration: by reducing T from 25 to 15, we speedup the decoding by 1.67× with only a minor performance degradation on SPICE (0.2%). We can acquire further speedup (2.1×) by setting T=12, with a small sacrifice on SPICE (0.3%). Nevertheless, the performance obtain by LaBERT with 10-20 refine steps are still competitive to the 25-step performance as well as the performance of the 4-level VLP, which verifies the capability of LaBERT in efficient image captioning decoding. Moreover, the global update rule and the [EOS] decay are shown to be important for LaBERT. After removing them, the CIDEr-D score of LaBERT on the second level drops by 1.6% and 2.2%, respectively.