Deep Bayesian Network for Visual Question Generation
Table 2: Comparison with state-of-the-art and different combination of Cues. The first block consists of the SOTA methods, second block depicts the models which uses only a single type of information such as Image or Place, third block has models which take one cue along with the Image information, fourth block takes two cues along with the Image information. The second last block consists of variations of our method. First is MC-SMN (Simple Moderator Network) in which there is no dropout (w/o Dropout) at inference time as explained in section 4.3 and the second one uses Gaussian dropout instead of the Bernoulli dropout (BD) which we have used across all the models.
['[BOLD] Methods', '[BOLD] BLEU1  [BOLD] Max', '[BOLD] BLEU1  [BOLD] Avg', '[BOLD] METEOR  [BOLD] Max', '[BOLD] METEOR  [BOLD] Avg']
The first analysis is considering the various combinations of cues such as caption and place. We use these models as our baseline and compare other variations of our model with the best single cue. The third block takes into consideration one cue along with the Image information, and we see an improvement of around 4% in BLEU1 and 2% in METEOR score. The fourth block takes two cues along with the Image information and obtains an improvement of around 10% in BLEU and 3% in METEOR scores. The question tags performs the best among all the 3 tags. This is reasonable as question tag can guide the type of question. The second last block consists of variations of our method. the first variation corresponds to the model in which there is no dropout at inference time and the second one uses Gaussian dropout instead of the Bernoulli dropout which we have used across all the models. As we can see, the application of dropout leads to a significant increase in the BLEU score and also Bernoulli dropout works best. We also observe that our proposed method MC-BMN gets an improvement of 13% in BLEU and 5% in METEOR score over the single cue baselines. Tags work well in general along with other cues than caption as it provides more precise information compared to the caption, but the performance drops significantly if only the tag information is provided as there is not much information for generating sensible questions. While comparing the various embedding, we also evaluated various ways of integrating the different cues to obtain joint embedding. We observe that in terms of METEOR score, obtain an improvement of around 3% using our proposed method over previous work by Mostafazadeh et. But it’s still quite far from human performance.