ChatGPT was actually finetuned by RLHF (reinforcement learning on human feedback), meaning it's trained in large extent to give responses that pleased its human interlocutors during training. The observatio