Universal Language Model Fine-tuning for Text Classification
Table 5: Validation error rates for ULMFiT with a vanilla LM and the AWD-LSTM LM.
['LM', 'IMDb', 'TREC-6', 'AG']
S5SS0SSS0Px3 Impact of LM quality Using our fine-tuning techniques, even a regular LM reaches surprisingly good performance on the larger datasets. On the smaller TREC-6, a vanilla LM without dropout runs the risk of overfitting, which decreases performance.