Contrastive Language Adaptation for Cross-Lingual Stance Detection
Table 2: Evaluation results on the target Arabic test dataset.
['[BOLD] Methods', '[BOLD] Weigh. Acc.', '[BOLD] Acc.', '[BOLD] Macro-F1', '[BOLD] F1 ( [ITALIC] agree, disagree, discuss, unrelated [BOLD] )']
S3SS0SSS0Px4 Results. The All-unrelated and All-agree baselines perform poorly across evaluation measures; All-unrelated performs better than All-agree because unrelated is the dominant class (∼68% of examples). It improves the performance of mono-lingual MN on weighted accuracy and F1, but its accuracy significantly drops. This is because adversarial approaches give higher weights to samples of the majority class (i.e., unrelated) which makes classification more challenging for the discriminator Montahaei et al. Also, although the agree and the discuss classes have roughly the same size, i.e., 474 and 409 examples, respectively, the results for agree are notably higher. This is mainly because the documents that discuss a claim often share the same topic with the claim, but they do not take a stance. In addition, the disagree examples are the most difficult ones; this class is by far the smallest one, with only 87 examples. We further estimate transferability by measuring the accuracy of the models in extracting evidence that support their predictions. A more accurate model should better transfer knowledge to the target language by accurately learning the relations between claims and pieces of evidence.