If ChatGPTs RL reward is calculated on entire output sequence as opposed to each token (current guess it is the former) output length, from reward perspective, seems to be tradeoff between SFT output length and output originality, which could