MultiFiT: Efficient Multi-lingual Language Model Fine-tuning
Table 5: Comparison of MultiFiT results with different pretraining corpora and ULMFiT, fine-tuned with 1k labels on MLDoc.
['[EMPTY]', 'de', 'es', 'zh']
Pretraining on more data generally helps. MultiFiT outperforms ULMFiT significantly; the performance improvement is particularly pronounced in Chinese where ULMFiT’s word-based tokenization underperformed.