Extending Multilingual BERT to Low-Resource Languages
Table 5: Comparison Between B-BERT vs E-MBERT: We compare the performance of E-MBERT with B-BERT on both English and target language. As a reference, performance of M-BERT is 79.37 on English. This shows that neither B-BERT nor E-MBERT gets unfair advantage from the English part of the model.
['[BOLD] Language', '[BOLD] English E-MBERT', '[BOLD] English  [BOLD] B-BERT', '[BOLD] Target  [BOLD] E-MBERT', '[BOLD] Target  [BOLD] B-BERT']
We can see that English performance of B-BERT is mostly better than Extend. However, in most cases Extend performs better on target language. This indicates that E-MBERT does not have an unfair advantage on English.