You can't beat it that way. I tried this with ChatGPT, and it gives a plausible explanation of its "reasoning." It is perfectly capable of outputting anything similar to its training set, which includes such explanations.