Zero-Shot Cross-lingual Classification Using Multilingual Neural Machine Translation
Table 7: Effect of parameter smoothing on the English SNLI test set and zero-shot performance on the French test set.
['Smoothing Range (steps)', 'SNLI (En)', 'SNLI (Fr)']
is a technique which aims to smooth point estimates of the learned parameters by averaging n steps from the training run and using it for inference. This is aimed at improving generalization and being less susceptible to the effects of over-fitting at inference. We hypothesize that a system with enhanced generalization might be better suited for zero-shot classification since it is a measure of the ability of the model to generalize to a new task.