Low-Resource Text Classification using Domain-Adversarial Learning
Table 4: Model accuracies using the monolingual and multilingual MUSE word vectors on the multilingual sentiment analysis tasks. For the Arabic Corpus, xtgt=500 labeled training samples were used in the low-resource setting (LR), while the CLS-10 task was given access to xtgt=2000 labeled elements. In the target-domain zero-shot scenario (ZS) and while training on the source domain only, no labeled training data from the target domain was used at all.
['Embedding', 'F', 'DANN [ITALIC] LR fastText', 'DANN [ITALIC] ZS fastText', 'S only fastText', 'DANN [ITALIC] LR MUSE', 'DANN [ITALIC] ZS MUSE']
Note that the monolingual word vectors are trained independently and therefore are (initially) unaligned in different subspaces. When using the averaging feature-extractor architecture (Favg) the results resemble those obtained by the ADAN model, indicating that simply fine-tuning the embeddings by learning a new projection matrix does not generally improve model performance. The models trained only in the source domain without domain-adversarial training are not able to learn a usable classification rule for the target domain. When projecting the word vectors in a common space during training, we obtain on-par performance to the model architecture with the pretrained multilingual MUSE word vectors. This can be explained by the fact that the alignment is learnt during the training of the classifier and is thus optimized to best suit the task of the classifier, while the multilingual word vectors may be trained towards a different objective. The monolingual word vectors do not show any visible alignment between words in different languages. Using our model, they were projected into a common space during training and have an overall lower average undirected Hausdorff distance. The Hausdorff distance is the maximum (supremum) distance from any point in one set to the nearest (infimum) point in another set where the euclidean metric is used as a distance measure between any two points d(x,y). Using the embeddings before and after projection into the common space of the source and target domain (¯¯¯¯¯Xsrc and ¯¯¯¯¯Xtgt respectivley), a lower Hausdorff distance between the two sets indicates a better alignment of the two embedding spaces.