"You have to realize, ChatGPT uses Transformer architectures that are pre-trained in this self-supervised manner," observed "Self-supervised-learning is something I've been advocating for a long time, even before OpenAI existed," he said.