OpenAI is generally quite secretive about this, but they already mentioned that ChatGPT integrates human feedback, so the model certainly learns from some supervision signals. The extent and effect of this are obviously hard to tell.