On Feature Normalization and Data Augmentation
Table 10: Ablation study on ImageNet with different λ and p (exchange probability) trained for 300 epochs.
['Model', '[ITALIC] λ', '[ITALIC] p', 'Top-1 / Top-5 Error(%)']
λ and 1−λ serve as the target interpolation weights of labels yA, yB, respectively. To explore the relationship between λ and model performance, we train a ResNet50 on ImageNet with λ∈{0.3,0.5,0.7,0.9} with on PONO. We observe that generally higher λ leads to lower error, probably because more information is captured in the normalized features than in the moments. After all, moments only capture general statistics, whereas the features have many channels and can capture texture information in great detail. We also investigate various values of the exchange probability p (for fixed λ=0.9), but on the ImageNet data p=1 (i.e. apply MoEx on every image) tends to perform best.