I'm reaalllly curious what kind of test suite OpenAI has to determine how good ChatGPTs answers remain after another round of RLHF. You could probably make ChatGPT really boring / uncreative / less useful while still doing ok on a bunch of factual testbeds.