Towards Causal VQA: Revealing and Reducing Spurious Correlations by Invariant and Covariant Semantic Editing
Table 8: IV-VQAAugmentation: numbers on the left side of the arrow denote the accuracy/flipping for models fine-tuned using just real data whereas numbers on the right side show the performance of models when finetuned with real+synthetic data
['[EMPTY]', 'CL (%)', 'SAAA (%)', 'SNMN (%)']
In the paper, we show a compact representation of the table by plotting the reduction in flips/improvement in accuracy for models finetuned using real+edit data relative to the models finetuned only using the real data. For instance, for question-type ‘is this a’ for CNN+LSTM: we see there is (12.72-9.77)/12.72 which is about 23% reduction in flips. These numbers show that using synthetic data always leads to a reduction in flips and in some cases- also results in improved accuracy on the original VQA set. n-1 consistent and accurate as well.