NB: ChatGPT is not wholly self-taught. In addition to self-supervised leaning on the internet (predicting next words) its also tuned on a dataset of human answers and then RL on human ranking of answers. Which makes this also a contrib