Id want to finetune in two cases (there are others) * specific output form factor (ChatGPT, style transfer, others) * knowledge you want really isnt in the pre-trained set (medical, financial, or just news)