For anyone doubting if fine-tuning a LLM is worth it, you should really take a look at how ChatGPT (InstructGPT) was built... it is a fine-tuned GPT-3 "These models have the same architecture, and differ only by the fact that InstructGPT is fine-tuned on our human data."