My understanding is that ChatGPT learns functions on language, and compositions of those functions, that serve next-token prediction. For example, here ChatGPT shows it can compose "capitalize" and "translate to Spanish" to transform input text as requested.