Component Analysis for Visual Question Answering Architectures
TABLE II: Experiment using different fusion strategies.
['[ITALIC] Embedding', 'RNN', 'Fusion', 'Training', 'Validation']
The best result is obtained using the element-wise multiplication. Such an approach functions as a filtering strategy that is able to scale down the importance of irrelevant dimensions from the visual-question feature vectors. In other words, vector dimensions with high cross-modal affinity will have their magnitudes increased, differently from the uncorrelated ones that will have their values reduced. Summation does provide the worst results overall, closely followed by the concatenation operator. Moreover, among all the fusion strategies used in this study, multiplication seems to ease the training process as it presents a much higher training set accuracy (≈11% improvement) as well.