Overcoming Statistical Shortcutsfor Open-ended Visual Counting
Table 2: Grounding ability of models trained on original TallyQA. We report standard AP and our GroundP metric on COCO-Grounding dataset (see supplementary materials for details). Our retrained Counter* (zhang2018counter, ) reaches a different balance between simple and complex sets.
['[EMPTY]', 'COCO-Grounding GroundP', 'COCO-Grounding AP', 'TallyQA Simple Acc.', 'TallyQA Simple RMSE', 'TallyQA Complex Acc.', 'TallyQA Complex RMSE']
S4SS3SSS0Px1 Comparison on COCO-Grounding We measure the grounding ability as a proxy to evaluate if models have learned the proper counting mechanism, and to assess the interpretability of our model. To this end, we specifically design a dataset named COCO-Grounding, and a grounding metric for open-ended counting called GroundP. Both are detailed in the supplementary material and are publicly available. Intuitively, the GroundP metric measures the proportion of the total score that is correctly located in the ground truth bounding boxes. We use a AP with a detection threshold of 0.2. Both models have been trained on the original TallyQA dataset. As expected, we report a lower accuracy since models that over-rely on statistical shortcuts are not penalized on this dataset. We report the best performances on grounding, and a gain of +24.3 and +9.1 points respectively over our retrained version of Counter. We also report a gain of +8.7 and +4.7 points respectively over our model optimized without entropy regularization.