DRAU: Dual Recurrent Attention Units for Visual Question Answering
Table 1: Simple Net model and ablation study of the DRAU model results on the VQA 2.0 validation split. The number of trainable parameters are shown for each model. IF indicates the type of image features the model uses. VG indicates whether the method uses external data augmentation from the Visual Genome dataset.
['VQA 2.0 Validation Split Model', 'VQA 2.0 Validation Split # Parameters', 'VQA 2.0 Validation Split IF', 'VQA 2.0 Validation Split Fusion', 'VQA 2.0 Validation Split VG', 'VQA 2.0 Validation Split All', 'VQA 2.0 Validation Split Y/N', 'VQA 2.0 Validation Split Num.', 'VQA 2.0 Validation Split Other']
We compare using convolution against our recurrent attention in a simple baseline model. This baseline VQA model uses the same question representation as described in the previous section and the ResNet global image features. The input features are simply concatenated and sent to a visual attention mechanism. The processed attention is fed to the reasoning module. To avoid any type of parameter advantage, both models were designed to have the same number of parameters approximately. We report the mean and standard deviation of the each model. While swapping convolutional text attention with RTAU reduces performance by 0.66%, using both RTAU and RVAU in the same model improves the overall performance by almost 2%. This might indicate that multimodal recurrent text attention thrives with certain architectural designs since it improves performance in our DRAU model.