How to Fine-Tune BERT for Text Classification?
Table 8: Test error rates (%) on five text classification datasets.
['Model', 'IMDb', 'Yelp P.', 'Yelp F.', 'AG', 'DBP']
This changes however with the task-specific further pre-training where even BERTBASE outperforms ULMFiT on all tasks. BERTLARGE fine-tuning with task-specific further pre-training achieves state-of-the-art results.