Diverse beam Search: Decoding Diverse Solutions from Neural Sequence Models
Table 1: Top: Oracle SPICE@k and distinct n-grams on the COCO image captioning task at B=20. While we report SPICE, we observe similar trends in other metrics (reported in the supplement). Bottom: Breakdown of results by difficulty class, highlighting the relative improvement over BS.
['[EMPTY]', '[BOLD] Method', '[BOLD] SPICE', '[BOLD] Oracle SPICE@k @5', '[BOLD] Oracle SPICE@k @10', '[BOLD] Oracle SPICE@k @20', '[BOLD] Distinct n-Grams n = 1', '[BOLD] Distinct n-Grams 2', '[BOLD] Distinct n-Grams 3', '[BOLD] Distinct n-Grams 4']
Our method produces significantly more distinct n-grams (almost 300% increase in the number of 4-grams produced) as compared to BS. We also note that our method tends to produce slightly longer captions compared to beam search on average. Moreover, on the PASCAL-50S test split we observe that DBS finds more likely top-1 solutions on average – DBS obtains a maximum log-probability of -6.53 as against -6.91 got by BS of same beam width. While the performance of DBS is guaranteed to be better than a BS of size \nicefracBG, this experimental evidence suggests that using DBS as a replacement to BS leads to better or at least comparable performance. Results by Image Complexity. For example, at Oracle Spice@20, DBS achieves significant improvements over BS of 0.67, 0.91, and 1.13 for Simple, Average, and Complex images respectively. While DBS improves over BS in all settings, complex images benefit even more from diversity-inducing inference than simple images.