On Feature Normalization and Data Augmentation
Table 8: Ablation study on different design choices.
['name', 'MoEx', 'Test Error']
In the previous section we have established that MoEx yields significant improvements across many tasks and model architectures. In this section we shed light onto which design choices crucially contribute to these improvements. The column titled MoEx indicates whether we performed moment exchange or not. Interpolated targets. We investigate two variants: λ=1, which corresponds to no label interpolation; MoEx with label smoothing (essentially assigning a small loss to all labels except yA). The last row corresponds to our proposed method, MoEx (λ=0.9).