Image-Question-Answer Synergistic Network for Visual Dialog
Table 2: Performance of our discriminative model on validation dataset.
['[BOLD] N', '[BOLD] M', '[ITALIC] τ', '[BOLD] MRR', '[BOLD] R@1', '[BOLD] R@5', '[BOLD] R@10', '[BOLD] Mean']
We conduct several ablation studies to verify the contribution of each module to our discriminative model. The performance improves as τ decays, because most candidate answers are easily discriminated. This kind of loss is reduced as τ decreases and is nearly zero when τ=0.25 , saturating our primary stage. This makes the model pay more attention to incorrect answers scored near or higher than the correct answer. The last five lines show the results of the synergistic stage with different settings. MRR of this stage drops at N=10 and M=10, as the top answers of the primary stage become stable after several epochs, resulting in the synergistic stage learning bias. Feeding more samples by increasing M improves performance to produce the best model at M=30, showing that synergy can learn a better representation of the image, question, and answer. What of interest is that the performance drops at M=40 or N=15, possibly because the selected answers become less relevant to the inputs, and the second stage model’s ability is to score nearly correct answers but is sensitive to unrelated ones. Conversely, it also addresses the importance of the primary stage to the synergistic stage. And the primary stage is also necessary to balance the memory cost, since each answer learns its own attention map and fusion with the image in the synergistic stage, while in the primary stage, only one attention map is required for the question and image.