Have gotten to play with ChatGPT a bit now, and WOW it is IMPRESSIVE I wonder to what extent RHLF affects overall performance, even ignoring alignment. Arguably this is in opposition to The Scaling Hypothesis - a new optimization algorithm was necessary to really improve on GPT3