An Empirical Study of Factors Affecting Language-Independent Models
Table 2: Accuracy results of Sentiment Analysis on English, Japanese and Chinese between monolingual BERT and LIM-H.
['Language', 'Monolingual-BERT', 'LIM-H']
By leveraging data from non-native languages, our LIM outperforms the English-only BERT model by 1.8% and the Japanese-only BERT model by 0.7%, but falls behind the Chinese-only BERT model by 1.2%. It should be noted that BERT specifically pre-trained the Chinese-only model to account for its unique character tokenization. Therefore, it is still very encouraging to see that our LIM is comparable to a specially customized monolingual BERT model.