Zero-Shot Cross-lingual Classification Using Multilingual Neural Machine Translation
Table 6: Zero-shot analyses of classifier network model capacity. The SNLI (Fr) results report the zero-shot performance.
['[BOLD] Encoder components', 'Simpler classifier SNLI (En)', 'Simpler classifier SNLI (Fr)', 'Complex classifier SNLI (En)', 'Complex classifier SNLI (Fr)']
S7SS0SSS0Px3 Effect of Encoder/Classifier Capacity We study the effect of the capacity of the two parts of our model on the final accuracies. Next, we experimented with only reusing different parts of the multilingual encoder in a bottom-up fashion. It can be seen that, as expected going from a simple linear classifier to a complex classifier significantly improves both English and zero-shot French performance on the SNLI tasks. However, even a simple linear classifier can achieve significant zero-shot performance when provided with rich enough encodings (49.66 to 61.61 accuracy). However, changing the encoder capacity tells an interesting story. As we selectively reuse parts of the encoder from the embedding layer to the top, we notice that the English performance only increases by about 2% whereas the zero-shot performance increases by about 18% at most in the complex classifier. This means that the additional layers in the encoder are essential for the proposed system to model a language agnostic representation (interlingua) which enables it to perform better zero-shot classification. Moreover, it should be noted that best zero-shot performance is obtained by using the complex classifier and up to layer 3 of the encoder. Although this gap is not big enough to be significant, we hypothesize that top layer of the encoder could be very specific to the MT task and hence might not be best suited for zero-shot classification.