KorNLI and KorSTS: New Benchmark Datasets for Korean Natural Language Understanding
Table 5: Test set scores for fine-tuned cross-encoding language models. KorNLI scores are accuracy (%) and KorSTS scores are 100 \times Spearman correlation. †To ensure comparability with XNLI, we only use the MNLI portion of the KorNLI dataset.
['[BOLD] Model', '[BOLD] # Params.', '[BOLD] KorNLI†', '[BOLD] KorSTS']
For KorNLI, we only train on the MNLI portion of the KorNLI dataset to ensure comparability with XNLI. Overall, despite being smaller, the Korean RoBERTa models outperform the XLM-R models for their respective sizes (base and large). For each model, the larger variant outperforms the base one, consistent with the findings in [liu2019roberta]. The large version of Korean RoBERTa performed the best for both KorNLI (83.67) and KorSTS (85.27) among all models tested.