Nice blog post (co-authored by a Brown PhD student!) explaining how systems like ChatGPT are built using clever combinations of unsupervised (for language), supervised (for preferences), and reinforcement (for applying the latter to the former) learning: