I completely agree with you that being able to provide feedback to the reward model would be a great feature for ChatGPT. It would allow us to explore less "optimal" results and navigate between multiple interesting regions, rather th