I dont get it. Yes, training an AI to avoid offensive language doesnt do anything to eg avoid mesaoptimisers. But the fact that they tried to train ChatGPT to avoid offensive language and couldnt is a bad sign for RLHF as an alignm