FlauBERT: Unsupervised Language Model Pre-training for French
Table 4: Results on the French PAWS-X dataset.
['Model', 'Accuracy']
One can observe that the monolingual French models perform only slightly better than the multilingual model mBERT, which could be attributed to the characteristics of the PAWS-X dataset. Containing samples with high lexical overlap ratio, this dataset has been proved to be an effective measure of model sensitivity to word order and syntactic structure [yang2019pawsx]. A multilingual model such as mBERT, therefore, could capture these features as well as a monolingual model.