Universal Language Model Fine-tuning for Text Classification
Table 4: Validation error rates for ULMFiT with and without pretraining.
['Pretraining', 'IMDb', 'TREC-6', 'AG']
We compare using no pretraining with pretraining on WikiText-103 Merity et al. Pretraining is most useful for small and medium-sized datasets, which are most common in commercial applications. However, even for large datasets, pretraining improves performance.