chatGPTinstructGPTinstructGPT models show promising generalization to instructions outside of the RLHF fine-tuning distribution.We qualitatively probe InstructGPTs capabilities, and find that it is abl