Say how you really feel ChatGPT... (I know this is still benign because it's only outputting langauge and not taking agentic actions in the real world but -- this out of distribution behavior is still happening after months of RLHF...makes you think about how hard alignment is)