XCOPA: A Multilingual Dataset for Causal Commonsense Reasoning
Table 6: XCOPA accuracy scores of different transfer variants that adapt to out-of-sample languages.
['[BOLD] Setup', '[BOLD] Model', 'ht', 'qu']
When using sufficiently large fine-tuning datasets (SI-ZS and SI+CO-ZS setups) all adaptation methods yield substantial improvements over the base XLM-R model. The improvements are less consistent in the COPA-ZS setup. However, we attribute it to the limited size of the English COPA dataset (only 400 instances) used for fine-tuning rather than to the ineffectiveness of the adaptation strategies. A comparison between XLMR+T and XLMR+S+T suggests that additional MLM pretraining on a moderately sized target language corpus does not lead to forgetting of the source language information. SIQA+COPA-ZS transfer setups. Further, the results with the most resource-lenient method (XLMR+D) also reveal positive trends. Further adaptation strategies (ponti2019towards) and downstream tasks warrant future investigation.