Extending Multilingual BERT to Low-Resource Languages
Table 1: Performance of M-BERT, Extend with LORELEI data and Extend with Wikipedia data.
['[BOLD] Lang', 'M-BERT', 'E w/ Lrl', 'E w/ Wiki']
The effectiveness of E-MBERT may be partially explained by the extra monolingual data the model is trained on. To explore the performance of E-MBERT without this extra training data, we Extend with using Wikipedia data, which is used in M-BERT.