An interesting failure mode of ChatGPT is rationalization. You can prompt model to reason out loud. There're two patterns of answer for that 1. Output reasoning, then answer. 2. Output answer, then reasoning. In (2) everything after answer is rationalization, not reasoning.