A little instruction is all you need? RL-based fine-tuning in InstructGPT takes ~1% of the compute of mask-language-modeling-pretraining, but it is what makes impressive performance of davinci-003 and ChatGPT possible.