"ChatGPT uses SSL much more than any kind of RL." - I agree with this 100%. At the same time I think that "amount" is less significant than the result. It seems to me that the RL/PPO component made a difference in this case.