With ChatGPT's mind blowing results, ML community is getting more curious about RLHF. RLHF outperforms Supervised Finetune (SFT) as shown in InstructGPT. But RLHF uses an extra large dataset in step 2. Thus, a missing baseline is SFT on both datasets from step 1 and 2. [1/2]