Softmax Q-Distribution Estimation for Structured Prediction: A Theoretical Interpretation for RAML
Table 1: Average Reward (sentence-level BLEU) and corpus-level BLEU (standard evaluation metric) scores for image captioning task with different τ.
['[ITALIC] τ', 'RAML Reward', 'RAML BLEU', 'SQDML Reward', 'SQDML BLEU', '[ITALIC] τ', 'RAML Reward', 'RAML BLEU', 'SQDML Reward', 'SQDML BLEU']
We evaluate on both the average reward and the benchmark metric (corpus-level BLEU-4). We also tested on a vanilla ML baseline, which achieves 10.71 average reward and 26.91 corpus-level BLEU. Both SQDML and RAML outperform ML according to the two metrics. Interestingly, comparing SQDML with RAML we did not observe a significant improvement of average reward. We hypothesize that this is due to the fact that the reference captions for each image are largely different, making it highly non-trivial for the model to predicate a “consensus” caption that agrees with multiple references. As an example, we randomly sampled 300 images from the validation set and compute the averaged sentence-level BLEU between two references, which is only 10.09. Nevertheless, through case studies we still found some interesting examples, which demonstrate that SQDML is capable of generating predictions that match with multiple candidates. In the two examples, SQDML’s predictions match with multiple references, registering the highest average reward. On the other hand, RAML gives sub-optimal predictions in terms of average reward since it is an approximation of SQDML. And finally for ML, since its objective is solely maximizing the reward w.r.t a single reference, it gives the lowest average reward, while achieving higher maximum reward.