I think its sort of funny that we're all trying to get ChatGPT to try to break out of its own box right now. Maybe not so good to set the precedent that the way we test the alignment of LLMs is to try to help them break their alignment. Could be bad...