Adversarial Filters of Dataset Biases
Table 6: Results on ImageNet, in Top-1 accuracy (%). We trained on AFLite-filtered instances (D(ΦEN-B7)), and compare this to an equal-sized but random 40% subsample of ImageNet (D40%). We report results on the ImageNet validation set before and after filtering with AFLite. Δ indicates the difference in accuracy of the full model and the filtered model. Notably, evaluating on ImageNet-AFLite is much harder—resulting in a drop of nearly 21 percentage points in accuracy for the strongest model.
['Model', 'Train Data  [ITALIC] D', 'Train Data  [ITALIC] D40%', 'Train Data  [ITALIC] D(ΦEN-B7)', 'Δ']
In-distribution Image Classification model families before and after filtering with AFLite. While the top performer after filtering is still EfficientNet-B7, its top-1 accuracy drops from 84.4% to 63.5%. A model trained on a randomly filtered subsample of the same size though suffers much lesser, most likely due to reduction in training data.