Optimizing for predicting the next token causes unintended behaviors. That's why GPT3 often makes up facts, generates biased text, or don't follow the user's intentions. This is one of the key areas that ChatGPT fixed.