This is behavior that is easy to get GPT-3 (the one that wasn't finetuned on human feedback) to stop doing; empirically ChatGPT does it much more, and responds stubbornly to attempts to get it to stop. So t